In reply to a tweet (local archive, Twitter) by @thingskatedid

Kate

@thingskatedid

First, lexing breaks up the input into a stream of tokens. PCRE has comments, (?#...)
That's information we can drop right there in the lexer. Read a comment, don't produce a token, and move on to read the next thing.