Python Tokens

Tokens

Tokens are the smallest unit of the program. There are following tokens in Python:

1. Keywords
2. Identifiers
3. Literals
4. Operators

Keywords

Keywords are a set of special words, which are reserved by python and they have some special meaning. we are not allowed to use these words as variable names, function names etc. Keywords are also case sensitive and all other words except True, False and None are in small case and they must be written as it is.List of Keywords are:

Keywords in Python

False	class	finally	is	return
None	continue	for	lambda	try
True	def	from	nonlocal	while
and	del	global	not	with
as	elif	if	or	yield
assert	else	import	pass
break	except	in	raise

Identifiers

Identifiers are nothing but user-defined names to represent programmable entity like variables, functions, classes, modules or any other objects.

The rules that we need to keep in mind while defining identifiers are:

1. You can use a sequence of letters (lowercase (a to z) or uppercase (A to Z)) or digits (0 to 9) or an underscore (_) while defining an identifier.
2. You can’t use digits to begin an identifier name.
3. You should not use Reserved Keywords to define an identifier.
4. Other than underscore (_) you are not allowed to use any other special characters i.e. special symbols like !, @, #, $, % etc. cannot be used in your identifier.
5. You can use Identifier of any length.

Literals

The other built-in objects in python are Literals. Literals can be defined as data that is given in a variable or constant. Python has following literals:

String Literals:A string literal is a sequence of characters surrounded by quotes. We can use both single, double or triple quotes for a string. And, a character literal is a single character surrounded by single or double quotes.
Numeric Literals:Numeric Literals are immutable (unchangeable). Numeric literals can belong to 3 different numerical types Integer, Float, and Complex.
Boolean Literals:A Boolean literal can have any of the two values: True or False.
Collection literals:There are four different literal collections List literals, Tuple literals, Dict literals, and Set literals.Special literals:Python contains one special literal i.e. None. We use it to specify to that field that is not created.

Operators

Operators are the symbols which perform the operation on some values and the values on which operations are performed are known as operands. We can categorize operators into the following categories in python:

1. Arithmetic Operators (+, -, *, /)
2. Relational Operators (<, >, <=, >=, ==, !=)
3. Assignment Operators (=, +=, -=, *=, /= etc)
4. Logical Operators (and, or, not)
5. Membership Operators (in, not in)
6. Identity Operators (is, is not)
7. Bitwise Operators (&, |, ~, ^ etc)

We will be discussing more about Operators in a separate article in details