Javacc Eof Token Code Hyper Link

With this option you can see how each character in the input stream is matched to create tokens.

I have used JavaCC in the past, but whenever I use it after a long gap, I have to relearn a few things about it particularly handling warnings.

So I thought this time I would blog about ways to handle some of the frequent warnings that I have seen.

You create grammer using EBNF ( Extended Backus-Naur Form ) and feed it to JavaCC.

There are some very good tutorials available at JavaCC Documentation page and FAQ.

I especially find Lookahead MiniTutorial and Token Manager MiniTutorial very useful.

Javacc Eof Token Code Hyper Link

If you use Eclipse IDE, then you would find JavaCC plugin for Eclipse useful it provides wizard to create JavaCC or JJTree (JJTree creates AST, Abstract Syntax Tree, after parsing the input) files, provides code colorization, outline, code hyper link, syntax checking and compilation.

You can also set JavaCC debug options easily using this plugin.

But when you run the parser on input 123, you will get error Exception in thread main java.lang.Error: Missing return statement in function.

In the grammer file, this object is returned at the end return jjtThis, but it returns only if the matched token is.

This is because Java block of return jjtThis is associated with.

We can fix this issue by grouping expression choices and associating Java block to the group.

JavaCC warning message, in fact, clearly indicates common token that caused this warning.

However fixing this warning may not be easy in a large and complex grammer.

So if we tell JavaCC to look for next two tokens before deciding which path to take (prod1 or prod2), then this warning will go away.

For example, feed an input of abc def to the above grammer and it will parse the input without any error.

However you would expect the parser to throw error in this case.

This can be fixed by telling JavaCC that input should end after either prod1 or prod2.

We can do this by adding token at the end of Start production.

However JavaCC would look next two tokens before deciding if prod1 or prod2 should be matched and would see that prod1 matches next two tokens and would select it for matching.

However after matching prod1, the parser expects end of input and when it finds integer 123, it throws error.

To see how token manager creates tokens from the input, set DEBUGTOKENMANAGER true.