I grew up in a family of compulsive storytellers and conversationalists who engaged in constant verbal one-upmanship, so I have an advantage when it comes to snark.
People ask me sometimes how one writes banter-filled dialog. I think it works about like this: it would be the same as an argument if it weren’t for the laughter. Like an argument, banter is competitive. But it resembles flirting because it’s cooperative at the same time. It’s a game, though often with enough seriousness that a misstep will turn it into an argument. That soupcon of danger helps make it good.
In fiction, characters who are attuned to each other can say almost anything to each other and get away with it because they understand what the other person means. In my in-progress novel, Jen Meets Her Match, Jen says “I hate you!” to her boyfriend Frank almost every day. By which she might mean almost anything.
I guess the main thing with banter, as opposed to, say, Spider-man mouthing off at people who don’t have the skill to hold up their end of the conversation, is that it takes two to banter. Some of the old screwball comedy movies, especially the Thin Man series, do very well with this. You also see good (if zany) examples in Duck Soup and other Marx Brothers movies, not to mention somewhat less ancient offerings like The Princess Bride.