torch¶
The torch package contains data structures for multi-dimensional tensors and defines mathematical operations over these tensors. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.
It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0
Tensors¶
is_tensor |
Returns True if obj is a PyTorch tensor. |
is_storage |
Returns True if obj is a PyTorch storage object. |
is_complex |
Returns True if the data type of |
is_floating_point |
Returns True if the data type of |
is_nonzero |
Returns True if the |
set_default_dtype |
Sets the default floating point dtype to |
get_default_dtype |
Get the current default floating point |
set_default_tensor_type |
Sets the default |
numel |
Returns the total number of elements in the |
set_printoptions |
Set options for printing. |
set_flush_denormal |
Disables denormal floating numbers on CPU. |
Creation Ops¶
Note
Random sampling creation ops are listed under Random sampling and
include:
torch.rand()
torch.rand_like()
torch.randn()
torch.randn_like()
torch.randint()
torch.randint_like()
torch.randperm()
You may also use torch.empty()
with the In-place random sampling
methods to create torch.Tensor
s with values sampled from a broader
range of distributions.
tensor |
Constructs a tensor with |
Constructs a sparse tensor in COO(rdinate) format with specified values at the given |
|
as_tensor |
Convert the data into a torch.Tensor. |
as_strided |
Create a view of an existing torch.Tensor |
from_numpy |
Creates a |
zeros |
Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument |
zeros_like |
Returns a tensor filled with the scalar value 0, with the same size as |
ones |
Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument |
ones_like |
Returns a tensor filled with the scalar value 1, with the same size as |
arange |
Returns a 1-D tensor of size
with values from the interval |
range |
Returns a 1-D tensor of size
with values from |
linspace |
Creates a one-dimensional tensor of size |
logspace |
Creates a one-dimensional tensor of size |
eye |
Returns a 2-D tensor with ones on the diagonal and zeros elsewhere. |
empty |
Returns a tensor filled with uninitialized data. |
empty_like |
Returns an uninitialized tensor with the same size as |
empty_strided |
Returns a tensor filled with uninitialized data. |
full |
Creates a tensor of size |
full_like |
Returns a tensor with the same size as |
quantize_per_tensor |
Converts a float tensor to a quantized tensor with given scale and zero point. |
quantize_per_channel |
Converts a float tensor to a per-channel quantized tensor with given scales and zero points. |
Returns an fp32 Tensor by dequantizing a quantized Tensor |
|
complex |
Constructs a complex tensor with its real part equal to |
polar |
Constructs a complex tensor whose elements are Cartesian coordinates corresponding to the polar coordinates with absolute value |
heaviside |
Computes the Heaviside step function for each element in |
Indexing, Slicing, Joining, Mutating Ops¶
cat |
Concatenates the given sequence of |
chunk |
Splits a tensor into a specific number of chunks. |
column_stack |
Creates a new tensor by horizontally stacking the tensors in |
dstack |
Stack tensors in sequence depthwise (along third axis). |
gather |
Gathers values along an axis specified by dim. |
hstack |
Stack tensors in sequence horizontally (column wise). |
index_select |
Returns a new tensor which indexes the |
masked_select |
Returns a new 1-D tensor which indexes the |
movedim |
Moves the dimension(s) of |
moveaxis |
Alias for |
narrow |
Returns a new tensor that is a narrowed version of |
nonzero |
|
reshape |
Returns a tensor with the same data and number of elements as |
row_stack |
Alias of |
scatter |
Out-of-place version of |
scatter_add |
Out-of-place version of |
split |
Splits the tensor into chunks. |
squeeze |
Returns a tensor with all the dimensions of |
stack |
Concatenates a sequence of tensors along a new dimension. |
swapaxes |
Alias for |
swapdims |
Alias for |
t |
Expects |
take |
Returns a new tensor with the elements of |
tensor_split |
Splits a tensor into multiple sub-tensors, all of which are views of |
tile |
Constructs a tensor by repeating the elements of |
transpose |
Returns a tensor that is a transposed version of |
unbind |
Removes a tensor dimension. |
unsqueeze |
Returns a new tensor with a dimension of size one inserted at the specified position. |
vstack |
Stack tensors in sequence vertically (row wise). |
Return a tensor of elements selected from either |
Generators¶
Generator |
Creates and returns a generator object that manages the state of the algorithm which produces pseudo random numbers. |
Random sampling¶
seed |
Sets the seed for generating random numbers to a non-deterministic random number. |
manual_seed |
Sets the seed for generating random numbers. |
initial_seed |
Returns the initial seed for generating random numbers as a Python long. |
get_rng_state |
Returns the random number generator state as a torch.ByteTensor. |
set_rng_state |
Sets the random number generator state. |
-
torch.
default_generator
Returns the default CPU torch.Generator¶
bernoulli |
Draws binary random numbers (0 or 1) from a Bernoulli distribution. |
multinomial |
Returns a tensor where each row contains |
Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. |
|
poisson |
Returns a tensor of the same size as |
rand |
Returns a tensor filled with random numbers from a uniform distribution on the interval |
rand_like |
Returns a tensor with the same size as |
randint |
Returns a tensor filled with random integers generated uniformly between |
randint_like |
Returns a tensor with the same shape as Tensor |
randn |
Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution). |
randn_like |
Returns a tensor with the same size as |
randperm |
Returns a random permutation of integers from |
In-place random sampling¶
There are a few more in-place random sampling functions defined on Tensors as well. Click through to refer to their documentation:
torch.Tensor.bernoulli_()
- in-place version oftorch.bernoulli()
torch.Tensor.cauchy_()
- numbers drawn from the Cauchy distributiontorch.Tensor.exponential_()
- numbers drawn from the exponential distributiontorch.Tensor.geometric_()
- elements drawn from the geometric distributiontorch.Tensor.log_normal_()
- samples from the log-normal distributiontorch.Tensor.normal_()
- in-place version oftorch.normal()
torch.Tensor.random_()
- numbers sampled from the discrete uniform distributiontorch.Tensor.uniform_()
- numbers sampled from the continuous uniform distribution
Quasi-random sampling¶
The |
Serialization¶
save |
Saves an object to a disk file. |
load |
Loads an object saved with |
Parallelism¶
get_num_threads |
Returns the number of threads used for parallelizing CPU operations |
set_num_threads |
Sets the number of threads used for intraop parallelism on CPU. |
get_num_interop_threads |
Returns the number of threads used for inter-op parallelism on CPU (e.g. |
set_num_interop_threads |
Sets the number of threads used for interop parallelism (e.g. |
Locally disabling gradient computation¶
The context managers torch.no_grad()
, torch.enable_grad()
, and
torch.set_grad_enabled()
are helpful for locally disabling and enabling
gradient computation. See Locally disabling gradient computation for more details on
their usage. These context managers are thread local, so they won’t
work if you send work to another thread using the threading
module, etc.
Examples:
>>> x = torch.zeros(1, requires_grad=True)
>>> with torch.no_grad():
... y = x * 2
>>> y.requires_grad
False
>>> is_train = False
>>> with torch.set_grad_enabled(is_train):
... y = x * 2
>>> y.requires_grad
False
>>> torch.set_grad_enabled(True) # this can also be used as a function
>>> y = x * 2
>>> y.requires_grad
True
>>> torch.set_grad_enabled(False)
>>> y = x * 2
>>> y.requires_grad
False
no_grad |
Context-manager that disabled gradient calculation. |
enable_grad |
Context-manager that enables gradient calculation. |
set_grad_enabled |
Context-manager that sets gradient calculation to on or off. |
Math operations¶
Pointwise Ops¶
abs |
Computes the absolute value of each element in |
absolute |
Alias for |
acos |
Computes the inverse cosine of each element in |
arccos |
Alias for |
acosh |
Returns a new tensor with the inverse hyperbolic cosine of the elements of |
arccosh |
Alias for |
Adds the scalar |
|
addcdiv |
Performs the element-wise division of |
addcmul |
Performs the element-wise multiplication of |
angle |
Computes the element-wise angle (in radians) of the given |
asin |
Returns a new tensor with the arcsine of the elements of |
arcsin |
Alias for |
asinh |
Returns a new tensor with the inverse hyperbolic sine of the elements of |
arcsinh |
Alias for |
atan |
Returns a new tensor with the arctangent of the elements of |
arctan |
Alias for |
atanh |
Returns a new tensor with the inverse hyperbolic tangent of the elements of |
arctanh |
Alias for |
atan2 |
Element-wise arctangent of with consideration of the quadrant. |
bitwise_not |
Computes the bitwise NOT of the given input tensor. |
bitwise_and |
Computes the bitwise AND of |
bitwise_or |
Computes the bitwise OR of |
bitwise_xor |
Computes the bitwise XOR of |
ceil |
Returns a new tensor with the ceil of the elements of |
clip |
Alias for |
conj |
Computes the element-wise conjugate of the given |
copysign |
Create a new floating-point tensor with the magnitude of |
cos |
Returns a new tensor with the cosine of the elements of |
cosh |
Returns a new tensor with the hyperbolic cosine of the elements of |
deg2rad |
Returns a new tensor with each of the elements of |
div |
Divides each element of the input |
divide |
Alias for |
digamma |
Computes the logarithmic derivative of the gamma function on input. |
erf |
Computes the error function of each element. |
erfc |
Computes the complementary error function of each element of |
erfinv |
Computes the inverse error function of each element of |
exp |
Returns a new tensor with the exponential of the elements of the input tensor |
exp2 |
Computes the base two exponential function of |
expm1 |
Returns a new tensor with the exponential of the elements minus 1 of |
fix |
Alias for |
float_power |
Raises |
floor |
Returns a new tensor with the floor of the elements of |
floor_divide |
|
fmod |
Computes the element-wise remainder of division. |
frac |
Computes the fractional portion of each element in |
imag |
Returns a new tensor containing imaginary values of the |
ldexp |
Multiplies |
lerp |
Does a linear interpolation of two tensors |
lgamma |
Computes the logarithm of the gamma function on |
log |
Returns a new tensor with the natural logarithm of the elements of |
log10 |
Returns a new tensor with the logarithm to the base 10 of the elements of |
log1p |
Returns a new tensor with the natural logarithm of (1 + |
log2 |
Returns a new tensor with the logarithm to the base 2 of the elements of |
logaddexp |
Logarithm of the sum of exponentiations of the inputs. |
logaddexp2 |
Logarithm of the sum of exponentiations of the inputs in base-2. |
logical_and |
Computes the element-wise logical AND of the given input tensors. |
logical_not |
Computes the element-wise logical NOT of the given input tensor. |
logical_or |
Computes the element-wise logical OR of the given input tensors. |
logical_xor |
Computes the element-wise logical XOR of the given input tensors. |
logit |
Returns a new tensor with the logit of the elements of |
hypot |
Given the legs of a right triangle, return its hypotenuse. |
i0 |
Computes the zeroth order modified Bessel function of the first kind for each element of |
igamma |
Computes the regularized lower incomplete gamma function: |
igammac |
Computes the regularized upper incomplete gamma function: |
Multiplies each element of the input |
|
multiply |
Alias for |
mvlgamma |
Computes the multivariate log-gamma function) with dimension element-wise, given by |
nan_to_num |
Replaces |
neg |
Returns a new tensor with the negative of the elements of |
negative |
Alias for |
nextafter |
Return the next floating-point value after |
polygamma |
Computes the
derivative of the digamma function on |
Takes the power of each element in |
|
rad2deg |
Returns a new tensor with each of the elements of |
real |
Returns a new tensor containing real values of the |
reciprocal |
Returns a new tensor with the reciprocal of the elements of |
remainder |
Computes the element-wise remainder of division. |
round |
Returns a new tensor with each of the elements of |
rsqrt |
Returns a new tensor with the reciprocal of the square-root of each of the elements of |
sigmoid |
Returns a new tensor with the sigmoid of the elements of |
sign |
Returns a new tensor with the signs of the elements of |
sgn |
For complex tensors, this function returns a new tensor whose elemants have the same angle as that of the elements of |
signbit |
Tests if each element of |
sin |
Returns a new tensor with the sine of the elements of |
sinc |
Computes the normalized sinc of |
sinh |
Returns a new tensor with the hyperbolic sine of the elements of |
sqrt |
Returns a new tensor with the square-root of the elements of |
square |
Returns a new tensor with the square of the elements of |
sub |
Subtracts |
subtract |
Alias for |
tan |
Returns a new tensor with the tangent of the elements of |
tanh |
Returns a new tensor with the hyperbolic tangent of the elements of |
true_divide |
Alias for |
trunc |
Returns a new tensor with the truncated integer values of the elements of |
xlogy |
Computes |
Reduction Ops¶
Returns the indices of the maximum value of all elements in the |
|
argmin |
Returns the indices of the minimum value(s) of the flattened tensor or along a dimension |
amax |
Returns the maximum value of each slice of the |
amin |
Returns the minimum value of each slice of the |
Tests if all elements in |
|
|
|
Returns the maximum value of all elements in the |
|
Returns the minimum value of all elements in the |
|
dist |
Returns the p-norm of ( |
logsumexp |
Returns the log of summed exponentials of each row of the |
Returns the mean value of all elements in the |
|
Returns the median of the values in |
|
Returns the median of the values in |
|
mode |
Returns a namedtuple |
norm |
Returns the matrix norm or vector norm of a given tensor. |
Returns the sum of all elements, treating Not a Numbers (NaNs) as zero. |
|
Returns the product of all elements in the |
|
Returns the q-th quantiles of all elements in the |
|
nanquantile |
This is a variant of |
Returns the standard-deviation of all elements in the |
|
Returns the standard-deviation and mean of all elements in the |
|
Returns the sum of all elements in the |
|
unique |
Returns the unique elements of the input tensor. |
unique_consecutive |
Eliminates all but the first element from every consecutive group of equivalent elements. |
Returns the variance of all elements in the |
|
Returns the variance and mean of all elements in the |
|
count_nonzero |
Counts the number of non-zero values in the tensor |
Comparison Ops¶
allclose |
This function checks if all |
argsort |
Returns the indices that sort a tensor along a given dimension in ascending order by value. |
eq |
Computes element-wise equality |
equal |
|
ge |
Computes element-wise. |
greater_equal |
Alias for |
gt |
Computes element-wise. |
greater |
Alias for |
isclose |
Returns a new tensor with boolean elements representing if each element of |
isfinite |
Returns a new tensor with boolean elements representing if each element is finite or not. |
isinf |
Tests if each element of |
isposinf |
Tests if each element of |
isneginf |
Tests if each element of |
isnan |
Returns a new tensor with boolean elements representing if each element of |
isreal |
Returns a new tensor with boolean elements representing if each element of |
kthvalue |
Returns a namedtuple |
le |
Computes element-wise. |
less_equal |
Alias for |
lt |
Computes element-wise. |
less |
Alias for |
maximum |
Computes the element-wise maximum of |
minimum |
Computes the element-wise minimum of |
fmax |
Computes the element-wise maximum of |
fmin |
Computes the element-wise minimum of |
ne |
Computes element-wise. |
not_equal |
Alias for |
sort |
Sorts the elements of the |
topk |
Returns the |
msort |
Sorts the elements of the |
Spectral Ops¶
stft |
Short-time Fourier transform (STFT). |
istft |
Inverse short time Fourier Transform. |
bartlett_window |
Bartlett window function. |
blackman_window |
Blackman window function. |
hamming_window |
Hamming window function. |
hann_window |
Hann window function. |
kaiser_window |
Computes the Kaiser window with window length |
Other Operations¶
atleast_1d |
Returns a 1-dimensional view of each input tensor with zero dimensions. |
atleast_2d |
Returns a 2-dimensional view of each each input tensor with zero dimensions. |
atleast_3d |
Returns a 3-dimensional view of each each input tensor with zero dimensions. |
bincount |
Count the frequency of each value in an array of non-negative ints. |
block_diag |
Create a block diagonal matrix from provided tensors. |
broadcast_tensors |
Broadcasts the given tensors according to Broadcasting semantics. |
broadcast_to |
Broadcasts |
broadcast_shapes |
Similar to |
bucketize |
Returns the indices of the buckets to which each value in the |
cartesian_prod |
Do cartesian product of the given sequence of tensors. |
cdist |
Computes batched the p-norm distance between each pair of the two collections of row vectors. |
clone |
Returns a copy of |
combinations |
Compute combinations of length of the given tensor. |
cross |
Returns the cross product of vectors in dimension |
cummax |
Returns a namedtuple |
cummin |
Returns a namedtuple |
cumprod |
Returns the cumulative product of elements of |
cumsum |
Returns the cumulative sum of elements of |
diag |
|
diag_embed |
Creates a tensor whose diagonals of certain 2D planes (specified by |
diagflat |
|
diagonal |
Returns a partial view of |
einsum |
Sums the product of the elements of the input |
flatten |
Flattens |
flip |
Reverse the order of a n-D tensor along given axis in dims. |
fliplr |
Flip tensor in the left/right direction, returning a new tensor. |
flipud |
Flip tensor in the up/down direction, returning a new tensor. |
kron |
Computes the Kronecker product, denoted by
, of |
rot90 |
Rotate a n-D tensor by 90 degrees in the plane specified by dims axis. |
gcd |
Computes the element-wise greatest common divisor (GCD) of |
histc |
Computes the histogram of a tensor. |
meshgrid |
Take tensors, each of which can be either scalar or 1-dimensional vector, and create N-dimensional grids, where the th grid is defined by expanding the th input over dimensions defined by other inputs. |
lcm |
Computes the element-wise least common multiple (LCM) of |
logcumsumexp |
Returns the logarithm of the cumulative summation of the exponentiation of elements of |
ravel |
Return a contiguous flattened tensor. |
renorm |
Returns a tensor where each sub-tensor of |
Repeat elements of a tensor. |
|
roll |
Roll the tensor along the given dimension(s). |
searchsorted |
Find the indices from the innermost dimension of |
tensordot |
Returns a contraction of a and b over multiple dimensions. |
trace |
Returns the sum of the elements of the diagonal of the input 2-D matrix. |
tril |
Returns the lower triangular part of the matrix (2-D tensor) or batch of matrices |
tril_indices |
Returns the indices of the lower triangular part of a |
triu |
Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices |
triu_indices |
Returns the indices of the upper triangular part of a |
vander |
Generates a Vandermonde matrix. |
view_as_real |
Returns a view of |
view_as_complex |
Returns a view of |
BLAS and LAPACK Operations¶
addbmm |
Performs a batch matrix-matrix product of matrices stored in |
addmm |
Performs a matrix multiplication of the matrices |
addmv |
Performs a matrix-vector product of the matrix |
addr |
Performs the outer-product of vectors |
baddbmm |
Performs a batch matrix-matrix product of matrices in |
bmm |
Performs a batch matrix-matrix product of matrices stored in |
chain_matmul |
Returns the matrix product of the 2-D tensors. |
cholesky |
Computes the Cholesky decomposition of a symmetric positive-definite matrix or for batches of symmetric positive-definite matrices. |
cholesky_inverse |
Computes the inverse of a symmetric positive-definite matrix
using its Cholesky factor
: returns matrix |
cholesky_solve |
Solves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix . |
dot |
Computes the dot product of two 1D tensors. |
eig |
Computes the eigenvalues and eigenvectors of a real square matrix. |
geqrf |
This is a low-level function for calling LAPACK directly. |
ger |
Alias of |
inner |
Computes the dot product for 1D tensors. |
inverse |
Takes the inverse of the square matrix |
det |
Calculates determinant of a square matrix or batches of square matrices. |
logdet |
Calculates log determinant of a square matrix or batches of square matrices. |
slogdet |
Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices. |
lstsq |
Computes the solution to the least squares and least norm problems for a full rank matrix of size and a matrix of size . |
lu |
Computes the LU factorization of a matrix or batches of matrices |
lu_solve |
Returns the LU solve of the linear system
using the partially pivoted LU factorization of A from |
lu_unpack |
Unpacks the data and pivots from a LU factorization of a tensor. |
matmul |
Matrix product of two tensors. |
matrix_power |
Returns the matrix raised to the power |
matrix_rank |
Returns the numerical rank of a 2-D tensor. |
matrix_exp |
Returns the matrix exponential. |
mm |
Performs a matrix multiplication of the matrices |
mv |
Performs a matrix-vector product of the matrix |
orgqr |
Computes the orthogonal matrix Q of a QR factorization, from the (input, input2) tuple returned by |
ormqr |
Multiplies mat (given by |
outer |
Outer product of |
pinverse |
Calculates the pseudo-inverse (also known as the Moore-Penrose inverse) of a 2D tensor. |
qr |
Computes the QR decomposition of a matrix or a batch of matrices |
solve |
This function returns the solution to the system of linear equations represented by and the LU factorization of A, in order as a namedtuple solution, LU. |
svd |
Computes the singular value decomposition of either a matrix or batch of matrices |
svd_lowrank |
Return the singular value decomposition |
pca_lowrank |
Performs linear Principal Component Analysis (PCA) on a low-rank matrix, batches of such matrices, or sparse matrix. |
symeig |
This function returns eigenvalues and eigenvectors of a real symmetric matrix |
lobpcg |
Find the k largest (or smallest) eigenvalues and the corresponding eigenvectors of a symmetric positive defined generalized eigenvalue problem using matrix-free LOBPCG methods. |
Estimate along dim, using the trapezoid rule. |
|
triangular_solve |
Solves a system of equations with a triangular coefficient matrix and multiple right-hand sides . |
vdot |
Computes the dot product of two 1D tensors. |
Utilities¶
compiled_with_cxx11_abi |
Returns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1 |
result_type |
Returns the |
can_cast |
Determines if a type conversion is allowed under PyTorch casting rules described in the type promotion documentation. |
promote_types |
Returns the |
use_deterministic_algorithms |
Sets whether PyTorch operations must use “deterministic” algorithms. |
are_deterministic_algorithms_enabled |
Returns True if the global deterministic flag is turned on. |
vmap |
vmap is the vectorizing map. |
_assert |
A wrapper around Python’s assert which is symbolically traceable. |