And the clincher which was that if given the choice between seeing his GC if he is a little grumpy or not seeing him at all he would prefer not to see him at all.
My father said this to me when our children were younger (and our elder son is now in his 40's). I was hurt and did not understand. But, today, I kind of "get it" even though I never would "hint" at such a thing to our children. Let me see if I can explain ....
I read somewhere that in my parents' generation, children were "seen and not heard". We raised our children "somewhere in the middle" and our grandchildren are being raised "to be front and center". No one way is correct; but this is what has occurred during the generations in my family. Today, my father is in his 90's and really prefers not to see or interact with any of his great grandchildren (he never was a "warm" father but I know he loves his children and grandchildren very much - frankly, the great grandchildren are "too much" for him).
Your IL's were wrong in criticizing you and your husband about your parenting style. But, I do know my father (and now deceased mother) could not have have endured the extra noise and interruption from a "grumpy, whinny child" in any of their grandparenting years (and they were "young" when they became grandparents) - due to their disposition and also the fact they lived (and continue to live) in a very quiet and silent place. Your IL's have apologized but I think what has happened here is the grandparents became overwhelmed with the noise and confusion - perhaps they are too "old" for a family barbeque (I so get this as I'm there now) and "lost it". I know our sons and their families just "don't get" that what they consider regular noise and and fun becomes overwhelming and exhausting confusion to us. When our children become adults and leave the house, and our home becomes quiet again, well - our tolerance levels change. And to make matters worse, we don't like admitting it and we hide our exhaustion (we like to pretend we're still young).
Give it time - your IL's love your/their family and I suspect they are trying to "cover" the exhaustion they feel when with your son because they don't want to give up any precious moments with him. I know, this is what I do. You probably have heard the saying "love to see them come and love to see them go" - my husband and I use this all of the time because it is true. And, we don't dare risk "telling the truth" because we might see our grandchildren less and that would be more painful. Anyway, just another perspective (maybe not even a correct one) but ....