C++ Tokens

Today, we are going to learn about C++ Tokens. Let us first define Tokens.

Tokens: 

The smallest, individual, and meaningful units of a program are called Tokens. Here meaningful means that they are well understood by Compiler. They are generally separated using spaces.

Types of Tokens:  

There are 6 types of tokens in C++.

1. Keywords                        2. Identifiers                        3. Constants or Literals

4. Operators                        5. Punctuators        6. Separators

We will discuss them all in this article.

1. Keywords:

- Keywords are the reserved words in any language. These are the tokens whose meaning is already explained to the compiler.

- A keyword cannot be used as an Identifier, i.e., it cannot be used to give the name to variables, function’s, classes and etc.

- For example, int, double, if, switch, auto etc. are keywords in C++.

- According to C++17, there are 73 tokens in C++, some of them are listed below:

2. Identifiers:

- Identifiers are the name given to a part of the program. The part may be a variable, a constant, a function name, a class name, etc.

- The sub-part of the program is accessed in the program using that identifier.

- There are certain rules for identifiers. These are given below:

i) An identifier cannot be a C++ keyword.

  For example, ‘auto’ is not a valid identifier, because it is a keyword in C++.

ii) An identifier can contain small letters, alphabet letters, numbers; but it cannot contain special characters and spaces (only ‘_’ can be used).

For example, Rahul123 is valid, but Rahul#123 is not a valid identifier.

iii) An identifier cannot start with a number; it must start with a letter or underscore ‘_’.

For example, 123Rahul is not valid, but _123Rahul and Rahul_123 are valid identifiers.

3. Constants or Literals:

- These are the token, whose value cannot be changed.

- They are also called as Literals.

There are several types of constants:

i) Integer Constants, these are the numbers without a decimal point. They are further classified as:

  A. Decimal Integer Constants: This includes 0 to 9 digits (Ten digits system) with optional positive or negative sign. For example: 10, -99 etc.

   B. Octal Integer Constants: these includes 0 to 7 (Eight Digits System) and starts with a ‘0’. For example: 026, 054 etc.

   C. Hexadecimal Integer Constant: these includes 0 to 9 and A to F digits (16 in total) digits and starts with ‘0x’. For example: 0x32, 0x2F etc. Generally, memory addresses are in Hexadecimal numbers.

ii) Floating Point Constants: these are the numbers with a decimal point and an optional positive-negative sign. For example, 2.3, 6.899 etc.

iii) Character Constants: these are the single characters enclosed within single quotes. For example, ‘a’, ‘Z’, ‘9’ etc.

iv) String Constants: string constants are the multiple characters enclosed within double quotes. For example, “Rahul”, “Programming”, “c”, etc.

4. Operators

-Operators are the tokens which operate on operands.

- Depending on the number of operands they are classified as Unary Operators (Operate on single operands like ++, -- etc.), Binary Operators (Operate on two operands like +,-,*, / etc.) and Ternary Operators (operates on Three operands like Conditional Operator “:?”).

- They are further classified on the basis of their type, but we will cover them in a separate article Operators.

- These are some common Operators in C++.

5. Punctuators: 

These are the tokens used for special use in a programming language. For example {,}, [,], (,) etc.

6. Separators: 

These are the tokens used to increase readability of program and separating tokens. This includes white spaces, tab spaces etc.

So, these are the tokens in C++. We will need them frequently from here onward.

_________________________________________

if any doubt or feedback please feel free to contact : 

    rahulrb15899@gmail.com

    +91 7775919753  (WhatsApp)