Anyone who works in machine learning will come across vectors. They can be used in different ways at different stages of a machine learning project, which can be confusing. Ultimately though, there are no machine learning models where vectors are not used.
Vectors In Machine Learning Models
There are three different stages that vectors can be used in machine learning:
- Input
Machines need input to be transformed into numbers, which then are represented as vectors. These can then be used to train models. In essence, they are ways of encoding information to become output.
- Model
Most machine learning projects have the goal of creating a model that performs a function. In deep learning models, this is achieved through neural networks.
- Output
Output of machine learning models can be in different formats, depending on the goal. These can be numbers, images or vectors. This is because a vector may sometimes represent the final outcome, and can be used to perform other operations or inputs. This can be relevant in Natural Language Processing models.
Vectors As Encoders: Input
When vectors are inputs, they are used to encode information in a format that your model can process. In order to train the model, we need to pass relevant information through it until it learns what it would look like. The model can then process this information, and the neural network can do operations on the input to tune the parameters; understanding what is and isn’t relevant.
The model can then be used, based on the information it’s learned. For example, this could be about house prices, certain song lyrics, or similar images. All extra information features help the machine to understand as best as it can. Through this a vector is created with a specific function.
In order for the data to be turned into vectors, it needs to be able to classify different kinds of information efficiently. One way it does this is by assigning numbers to different inputs. For example with language, certain words could be defined as numbers. With images, different colors or textures could be.
Vectors As Transformers: Model
By now, we’ve seen how it’s possible to represent input as a vector. It means that we need models to learn to put features into a certain format to create the desired output. Neural networks create models which can learn the functions.
Hidden layers of neural networks take in vectorized inputs, add weight to these (create a weight matrix), and then create desired outputs.
Vectors inhabit vector spaces, and are able to be manipulated to move around the space. Each vector space holds properties that define how each operation can take place within them. Vector spaces are important for machine learning, as this is where transformations (turning input into output) happen.
Vector As Operators: Output
Outputs of models can be images, texts, numbers or vectors. When the output is a vector, they will be serving two functions:
- Vector input to other models.
- Vector input to other vector functions. This means the output will be used with another vector function. This vector output does not make sense alone, but it needs to be combined with another function to interpret the result.
Conclusion
This has been a brief introduction to how vectors are used in machine learning. To take a deeper look at this topic, read this post which goes into more detail.
From this post we have seen that the use of vectors in machine learning is very specific. Vectors are used in different ways depending on whether they are input or output of models.