Subtopic 1 of 6
2.1Tokens, lexemes and patterns
A lexeme is the actual character sequence, a pattern (regular expression) describes a class, and a token is the pair <name, attribute>. Whitespace and comments are discarded, they are not tokens.
Rules in priority order: KEYWORD = if|else|while|int · ID = [A-Za-z_]\w* · NUM = \d+(\.\d+)? · RELOP = <=|>=|==|<|> · ASSIGN = = · OP = [-+*/] · PUNCT = [(){};,]
- at "if (coun": KEYWORD→"if", ID→"if". Longest wins (tie: first listed rule wins): <KEYWORD, if>
- at "(count <": PUNCT→"(". Longest wins: <PUNCT, (>
- at "count <=": ID→"count". Longest wins: <ID, count>
- at "<= 10) x": RELOP→"<=". Longest wins: <RELOP, <=>
- at "10) x = ": NUM→"10". Longest wins: <NUM, 10>
- at ") x = x ": PUNCT→")". Longest wins: <PUNCT, )>
- at "x = x + ": ID→"x". Longest wins: <ID, x>
- at "= x + 1;": ASSIGN→"=". Longest wins: <ASSIGN, =>
- at "x + 1;": ID→"x". Longest wins: <ID, x>
- at "+ 1;": OP→"+". Longest wins: <OP, +>
- at "1;": NUM→"1". Longest wins: <NUM, 1>
- at ";": PUNCT→";". Longest wins: <PUNCT, ;>
Previous year questions: Lexical Analysis (2)
PYQs are grouped by topic, so this is the full set for Lexical Analysis; try the ones that match this subtopic. Open all 2 on the topic page.