5 Interesting PyTorch Functions for beginners

Priyansh Warke

Dec 8, 2020·5 min read

An short introduction about PyTorch and the chosen functions.

• erfinv
• linspace
• transpose
• quasirandom.SobolEngine
• lgamma

What do you want to know about Pytorch?

PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook’s AI Research lab (FAIR). It is free and open-source software released under the Modified BSD license.

How do you install Pytorch?

For windows,

!pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

For MAC OS,

!pip install numpy torch torchvision torchaudio

For Linux,

!pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

How to import Pytorch?

To import PyTorch,

import torch

Function 1 — torch.erfinv

torch.erfinv is one of the math operations which computes the inverse error function of each element of output.

The inverse error function is defined in the range of (-1,1) as:

erfinv(erf(x))=x

# Example 1 - working (change this)
torch.tensor([0,0.2,-0.94])
torch.erfinv(torch.tensor([0,0.2,-0.94]))

Out[]:

tensor([ 0.0000,  0.1791, -1.3299])

Here, it is showing the output after applying erfinv fuction.

In [ ]:

# Example 2 - working
torch.erfinv(torch.tensor([0,0.01,-0.33333]))

Out[]:

tensor([ 0.0000,  0.0089, -0.3046])

The above example are showing output after applying erfinv function.

In[]:

# Example 3 - breaking (to illustrate when it breaks)
torch.erfinv(torch.tensor([1,3,4]))

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last)

<ipython-input-23-13f84a2ed394> in <module>()

1 # Example 3 - breaking (to illustrate when it breaks) ---->

2 torch.erfinv(torch.tensor([1,3,4]))

RuntimeError: erfinv_vml_cpu not implemented for 'Long'

The values in tensor functions are yielding Long Tensors which can not be interpreted by the torch.erfinv function. It can be solved by entering at least one value as a float. for eg.- 1 as 1.0 .

Function 2 — linspace

linspace function creates a one-dimensional tensor of size steps whose values are evenly spaced from start to end, inclusive. That is, the value are:

(start,start+end-start/steps,…,start +(steps-1)*end-start/steps, end)

In [5]:

# Example 1 -
import torch
torch.linspace(5,2,steps=6)

Out[5]:

tensor([5.0000, 4.4000, 3.8000, 3.2000, 2.6000, 2.0000])

Here we have a 1-dimensionl array as the output where the element’s starting value is 5,and then it is depreciating towards the value of 2 in 5 steps, and the middle values are shown according to the steps given in linspace function.

# Example 2 -
torch.linspace(-5,end=-15,steps=20)

Out[16]:

tensor([ -5.0000,  -5.5263,  -6.0526,  -6.5789,  -7.1053,  -7.6316,  -8.1579,
         -8.6842,  -9.2105,  -9.7368, -10.2632, -10.7895, -11.3158, -11.8421,
        -12.3684, -12.8947, -13.4211, -13.9474, -14.4737, -15.0000])

We can also get a 1-dimensional array with negative values by giving starting value negative and end value also negative if you want.

# Example 3 - breaking (to illustrate when it breaks)
torch.linspace(2,50,steps=-20)

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last)

<ipython-input-23-6970e8d7033d> in <module>()

1 # Example 3 - breaking (to illustrate when it breaks) ---->

2 torch.linspace(2,50,steps=-20)

RuntimeError: number of steps must be non-negative

In the above example, it was shown that 1-D array can be created with negative values by specifying start point and point with negative values.But,same is not applicable for giving a negative value in steps parameter.

It is not advisable to skip giving value to step parameter as it is possible that in future PyTorch versions, the output will give error.

The linspace function can be used for deep neutral network and nlp purposes.

Function 3 — transpose

torch.transpose(input,dim0,dim1)

Transpose function returns a tensor with a transposed version of input. The resulting out tensor shares it’s underlying storage with the input tensor, so changing the values would change the value of the other.

In [52]:

# Example 1 - working
x=torch.randn(3,4)
torch.transpose(x,-2,1)

Out[52]:

tensor([[ 0.3468,  0.5118, -1.9654],
        [ 0.5505, -0.4011, -0.8231],
        [-1.6721,  0.1320, -0.8271],
        [ 1.3627,  2.0719,  1.5848]])

The Output is showing a tensor with transposed version of x tensor and also the dimensions are swapped.

# Example 2 - working
x=torch.randn(1,4)
torch.transpose(x,0,1)

Out[54]:

tensor([[-1.4462],
        [-1.0067],
        [-1.9261],
        [ 0.4800]])

Explanation about example

In [62]:

# Example 3 - breaking (to illustrate when it breaks)
z=torch.randn(3,1,2)
torch.transpose(z,2,3)

--------------------------------------------------------------------------- IndexError Traceback (most recent call last)

<ipython-input-62-1581f2897b4e> in <module>()

1 # Example 3 - breaking (to illustrate when it breaks)

2 z=torch.randn(3,1,2) ----> 3 torch.transpose(z,2,3)

IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)

In the above example, index error was shown as dim1 parameter was not in the expected range.

The Transpose function can be used during matrices multiplication.

Function 4 — sobolengine

torch.quasirandom.SobolEngine(dimension,scramble=False,seed=None)

This function is an engine for generating(scrambled) Sobol sequences which are an example of low discrepancy quasi-random sequences.

In [65]:

# Example 1 - working
soboleng=torch.quasirandom.SobolEngine(dimension=3)
soboleng.draw(2)

Out[65]:

tensor([[0.5000, 0.5000, 0.5000],
        [0.7500, 0.2500, 0.7500]])

In the above example, the output is showing sobol sequences in the tensor with dimensions acting as number of columns.

# Example 2 - working
soboleng=torch.quasirandom.SobolEngine(4,scramble=True)
soboleng.draw(2)

Out[80]:

tensor([[0.6520, 0.9353, 0.0630, 0.6850],
        [0.7976, 0.2962, 0.3177, 0.3711]])

In the above example,we have produced a scrambled Sobol sequences with the scramble parameter.

In [82]:

# Example 3 - breaking (to illustrate when it breaks)
soboleng=torch.quasirandom.SobolEngine(0)
soboleng.draw(3)

--------------------------------------------------------------------------- ValueError Traceback (most recent call last)

<ipython-input-82-da5ae28173d6> in <module>()

1 # Example 3 - breaking (to illustrate when it breaks)

----> 2 soboleng=torch.quasirandom.SobolEngine(0)

3 soboleng.draw(3)

/usr/local/lib/python3.6/dist-packages/torch/quasirandom.py in __init__(self, dimension, scramble, seed)

46 def __init__(self, dimension, scramble=False, seed=None):

47 if dimension > self.MAXDIM or dimension < 1:

---> 48 raise ValueError("Supported range of dimensionality "

49 f"for SobolEngine is [1, {self.MAXDIM}]")

50 ValueError: Supported range of dimensionality for SobolEngine is [1, 1111]

The output is showing a ValueError as the dimension 0 is not acceptable in the supported range of dimensionality which is [1,1111].

The sobolengine can be used for sampling sequences upto a maximum dimension of 1111.

Function 5 — lgamma

torch.lgamma(input,*,out=None)

The lgamma function is used to compute the logarithm of the gamma function on input.

In [86]:

# Example 1 - working
x=torch.arange(-2.1,2.0,2.1)
torch.lgamma(x)

Out[86]:

tensor([1.5317,    inf])

In the above example, the output is showing the values of logarithm of the gamma function on x tensor.

In [103]:

# Example 2 - working
b=torch.arange(1.0,6.,0.2)
torch.lgamma(b)

Out[103]:

tensor([ 0.0000, -0.0854, -0.1196, -0.1126, -0.0711,  0.0000,  0.0969,  0.2169,
         0.3574,  0.5167,  0.6931,  0.8854,  1.0923,  1.3129,  1.5463,  1.7918,
         2.0486,  2.3161,  2.5939,  2.8813,  3.1781,  3.4836,  3.7977,  4.1199,
         4.4499])

In the above example, the output is showing the values of logarithms of the gamma function of b tensor.

In [113]:

# Example 3 - breaking (to illustrate when it breaks)
z=torch.arange(-1.0,3,1.0)
torch.lgamma(z^2)

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last)

<ipython-input-113-af9c7eaf8101> in <module>()

1 # Example 3 - breaking (to illustrate when it breaks)

2 z=torch.arange(-1.0,3,1.0) ----> 3 torch.lgamma(z^2)

RuntimeError: "bitwise_xor_cpu" not implemented for 'Float'

In the above example, the output is showing RuntimeError as the bitwise operation is not implemented for float

The lgamma function can be used to support broadcasting to a common shape and float inputs.

Conclusion

In this session, we have covered 5 different types of Pytorch functions for beginners. There are more functions which you can explore through the official Pytorch website. Also, It would be advisable to run the code yourself after reading the story.

Reference Links

Provide links to your references and other interesting articles about tensors

• Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
• https://en.wikipedia.org/wiki/PyTorch

Also, any Feedback would be much appreciated.