Lexical Syntax

Scala programs are written using the Unicode Basic Multilingual Plane (BMP) character set; Unicode supplementary characters are not presently supported.
This chapter defines the two modes of Scala's lexical syntax, the Scala mode and the XML mode.
If not otherwise mentioned, the following descriptions of Scala tokens refer to Scala mode, and literal characters (marked as c) refer to the ASCII fragment \u0000 – \u007F.

In Scala mode, Unicode escapes are replaced by the corresponding Unicode character with the given hexadecimal code.

UnicodeEscape ::= ‘\’ ‘u’ {‘u’} hexDigit hexDigit hexDigit hexDigit
hexDigit      ::= ‘0’ | … | ‘9’ | ‘A’ | … | ‘F’ | ‘a’ | … | ‘f’

To construct tokens, characters are distinguished according to the following classes (Unicode general category given in parentheses):

1. Whitespace characters. \u0020 | \u0009 | \u000D | \u000A.
2. Letters, which include lower case letters (Ll), upper case letters (Lu), titlecase letters (Lt), other letters (Lo), letter numerals (Nl) and the two characters \u0024 "$" and \u005F "_".
3. Digits "0" | … | "9".
4. Parentheses "(" | ")" | "[" | "]" | "{" | "}".
5. Delimiter characters "`" | "'" | """ | "." | ";" | ",".
6. Operator characters. These consist of all printable ASCII characters (\u0020 - \u007E) that are in none of the sets above, mathematical symbols (Sm) and other symbols (So).

Identifiers

op          ::=  opchar {opchar}
varid       ::=  lower idrest
boundvarid  ::=  varid
              | ‘`’ varid ‘`’
plainid     ::=  upper idrest
              |  varid
              |  op
id          ::=  plainid
              |  ‘`’ { charNoBackQuoteOrNewline | UnicodeEscape | charEscapeSeq } ‘`’
idrest      ::=  {letter | digit} [‘_’ op]

There are three ways to form an identifier.
First, an identifier can start with a letter which can be followed by an arbitrary sequence of letters and digits. This may be followed by underscore ‘_‘ characters and another string composed of either letters and digits or of operator characters.
Second, an identifier can start with an operator character followed by an arbitrary sequence of operator characters. The preceding two forms are called plain identifiers.
Finally, an identifier may also be formed by an arbitrary string between back-quotes (host systems may impose some restrictions on which strings are legal for identifiers). The identifier then is composed of all characters excluding the backquotes themselves.

As usual, a longest match rule applies. For instance, the string big_bob++=`def` decomposes into the three identifiers big_bob, ++=, and def.

The rules for pattern matching further distinguish between variable identifiers, which start with a lower case letter, and constant identifiers, which do not. For this purpose, underscore ‘_‘ is taken as lower case, and the ‘$’ character is taken as upper case.

The ‘$’ character is reserved for compiler-synthesized identifiers. User programs should not define identifiers which contain ‘$’ characters.

The following names are reserved words instead of being members of the syntactic class id of lexical identifiers.

abstract    case        catch       class       def
do          else        extends     false       final
finally     for         forSome     if          implicit
import      lazy        macro       match       new
null        object      override    package     private
protected   return      sealed      super       this
throw       trait       try         true        type
val         var         while       with        yield
_    :    =    =>    <-    <:    <%     >:    #    @

The Unicode operators \u21D2 $\Rightarrow$ and \u2190 $\leftarrow$, which have the ASCII equivalents => and <-, are also reserved.

Here are examples of identifiers:

x         Object        maxIndex   p2p      empty_?
+         `yield`       αρετη     _y       dot_product_*
__system  _MAX_LEN_

Copy

Newline Characters

semi ::= ‘;’ |  nl {nl}

Scala is a line-oriented language where statements may be terminated by semi-colons or newlines. A newline in a Scala source text is treated as the special token ‘nl’ if the three following criteria are satisfied:

1. The token immediately preceding the newline can terminate a statement.
2. The token immediately following the newline can begin a statement.
3. The token appears in a region where newlines are enabled.

The tokens that can terminate a statement are: literals, identifiers and the following delimiters and reserved words:

this    null    true    false    return    type    <xml-start>
_       )       ]       }

The tokens that can begin a statement are all Scala tokens except the following delimiters and reserved words:

catch    else    extends    finally    forSome    match
with    yield    ,    .    ;    :    =    =>    <-    <:    <%
>:    #    [    )    ]    }

A case token can begin a statement only if followed by a class or object token.

Newlines are enabled in:

1. all of a Scala source file, except for nested regions where newlines are disabled
2. the interval between matching { and } brace tokens, except for nested regions where newlines are disabled

Newlines are disabled in:

1. the interval between matching ( and ) parenthesis tokens, except for nested regions where newlines are enabled
2. the interval between matching [ and ] bracket tokens, except for nested regions where newlines are enabled
3. the interval between a case token and its matching => token, except for nested regions where newlines are enabled
4. any regions analyzed in XML mode

Note that the brace characters of {...} escapes in XML and string literals are not tokens, and therefore do not enclose a region where newlines are enabled.

Normally, only a single nl token is inserted between two consecutive non-newline tokens which are on different lines, even if there are multiple lines between the two tokens. However, if two tokens are separated by at least one completely blank line (i.e a line which contains no printable characters), then two nl tokens are inserted.

The Scala grammar (given in full here) contains productions where optional nl tokens, but not semicolons, are accepted. This has the effect that a newline in one of these positions does not terminate an expression or statement. These positions can be summarized as follows:

Multiple newline tokens are accepted in the following places (note that a semicolon in place of the newline would be illegal in every one of these cases):

• between the condition of a conditional expression or while loop and the next following expression
• between the enumerators of a for-comprehension and the next following expression
• after the initial type keyword in a type definition or declaration

A single new line token is accepted

• in front of an opening brace ‘{’, if that brace is a legal continuation of the current statement or expression
• after an infix operator, if the first token on the next line can start an expression
• in front of a parameter clause
• after an annotation

The newline tokens between the two lines are not treated as statement separators.

if (x > 0)
  x = x - 1

while (x > 0)
  x = x / 2

for (x <- 1 to 10)
  println(x)

type
  IntList = List[Int]

Copy

new Iterator[Int]
{
  private var x = 0
  def hasNext = true
  def next = { x += 1; x }
}

Copy

With an additional newline character, the same code is interpreted as an object creation followed by a local block:

new Iterator[Int]

{
  private var x = 0
  def hasNext = true
  def next = { x += 1; x }
}

Copy

x < 0 ||
x > 10

Copy

With an additional newline character, the same code is interpreted as two expressions:

x < 0 ||

x > 10

Copy

def func(x: Int)
        (y: Int) = x + y

Copy

With an additional newline character, the same code is interpreted as an abstract function definition and a syntactically illegal statement:

def func(x: Int)

        (y: Int) = x + y

Copy

@serializable
protected class Data { ... }

Copy

With an additional newline character, the same code is interpreted as an attribute and a separate statement (which is syntactically illegal).

@serializable

protected class Data { ... }

Copy

Literals

There are literals for integer numbers, floating point numbers, characters, booleans, symbols, strings. The syntax of these literals is in each case same as in Java.

Literal  ::=  [‘-’] integerLiteral
          |  [‘-’] floatingPointLiteral
          |  booleanLiteral
          |  characterLiteral
          |  stringLiteral
          |  interpolatedString
          |  symbolLiteral
          |  ‘null’

integerLiteral  ::=  (decimalNumeral | hexNumeral)
                      [‘L’ | ‘l’]
decimalNumeral  ::=  ‘0’ | nonZeroDigit {digit}
hexNumeral      ::=  ‘0’ (‘x’ | ‘X’) hexDigit {hexDigit}
digit           ::=  ‘0’ | nonZeroDigit
nonZeroDigit    ::=  ‘1’ | … | ‘9’

floatingPointLiteral  ::=  digit {digit} ‘.’ digit {digit} [exponentPart] [floatType]
                        |  ‘.’ digit {digit} [exponentPart] [floatType]
                        |  digit {digit} exponentPart [floatType]
                        |  digit {digit} [exponentPart] floatType
exponentPart          ::=  (‘E’ | ‘e’) [‘+’ | ‘-’] digit {digit}
floatType             ::=  ‘F’ | ‘f’ | ‘D’ | ‘d’

booleanLiteral  ::=  ‘true’ | ‘false’

characterLiteral  ::=  ‘'’ (charNoQuoteOrNewline | UnicodeEscape | charEscapeSeq) ‘'’

stringLiteral  ::=  ‘"’ {stringElement} ‘"’
stringElement  ::=  charNoDoubleQuoteOrNewline | UnicodeEscape | charEscapeSeq

stringLiteral   ::=  ‘"""’ multiLineChars ‘"""’
multiLineChars  ::=  {[‘"’] [‘"’] charNoDoubleQuote} {‘"’}

interpolatedString  ::= alphaid ‘"’ {printableChar (‘"’ | ‘$’) | escape} ‘"’
                      |  alphaid ‘"""’ {[‘"’] [‘"’] char (‘"’ | ‘$’) | escape} {‘"’} ‘"""’
escape              ::= ‘$$’
                      | ‘$’ id
                      | ‘$’ BlockExpr
alphaid             ::= upper idrest
                      |  varid

implicit class StringInterpolation(s: StringContext) {
  def id(args: Any*) = ???
}

symbolLiteral  ::=  ‘'’ plainid

package scala
final case class Symbol private (name: String) {
  override def toString: String = "'" + name
}

Whitespace and Comments

Tokens may be separated by whitespace characters and/or comments. Comments come in two forms:

A single-line comment is a sequence of characters which starts with // and extends to the end of the line.

A multi-line comment is a sequence of characters between /* and */. Multi-line comments may be nested, but are required to be properly nested. Therefore, a comment like /* /* */ will be rejected as having an unterminated comment.

Trailing Commas in Multi-line Expressions

If a comma (,) is followed immediately, ignoring whitespace, by a newline and a closing parenthesis ()), bracket (]), or brace (}), then the comma is treated as a "trailing comma" and is ignored. For example:

foo(
  23,
  "bar",
  true,
)

XML mode

In order to allow literal inclusion of XML fragments, lexical analysis switches from Scala mode to XML mode when encountering an opening angle bracket ‘<’ in the following circumstance: The ‘<’ must be preceded either by whitespace, an opening parenthesis or an opening brace and immediately followed by a character starting an XML name.

( whitespace | ‘(’ | ‘{’ ) ‘<’ (XNameStart | ‘!’ | ‘?’)

  XNameStart ::= ‘_’ | BaseChar | Ideographic // as in W3C XML, but without ‘:’

The scanner switches from XML mode to Scala mode if either of these is true:

• the XML expression or the XML pattern started by the initial ‘<’ has been successfully parsed
• the parser encounters an embedded Scala expression or pattern and forces the Scanner back to normal mode, until the Scala expression or pattern is successfully parsed. In this case, since code and XML fragments can be nested, the parser has to maintain a stack that reflects the nesting of XML and Scala expressions adequately

Note that no Scala tokens are constructed in XML mode, and that comments are interpreted as text.

The following value definition uses an XML literal with two embedded Scala expressions:

val b = <book>
          <title>The Scala Language Specification</title>
          <version>{scalaBook.version}</version>
          <authors>{scalaBook.authors.mkList("", ", ", "")}</authors>
        </book>

Copy