Python常用库之Numpy | Felix’s Micro Space

type

date

slug

A: CHEAT SHEET

B: Help System

Stage 1 — Quick Introduction

1.1 Highlighted Introduction

In Contrast with Other Modules or Progamming Language

Important distinction between numpy array and native python array

Numpy Arrays have a fixed size unlike Python lists with dynamic lengh. The modification of length of array will delete original array.

Numpy arrays operation own higher execution efficiency with less code than is possible using python's built-in sequences(list)

More and more mathmatical packages make numpy array as the priorial choice instead of native code

In Contrast with python list, python array

python list support the input of different datatype, and this means it will decrease the efficiency of operation

python array is fit with the only one datatype

Benefit and Application

For-loop method of element operation will result in lower efficency and higher nested complexity. (we will pay the prices for the inefficencies of looping in python). we could accomplish the same work much more easily in C by writing beacuse this will save the overhead involved in interpreting the python code and manipuating Python objects. Numpy gives us the best of both altenatives(worlds):

Element-by-elemnt operation are the "the default mode" when the ndarry is involve, but he element-by-elment operation is speedily execuated by pre-compiled C codd;

do at near-C speed, but with the code simplicity we expect from something based on python

Vectorization describe the absence of any explicit looping

vectorized code is more concise and easier to read

fewer lines of code generally means fewer bugs

the code more closely resembles standard mathmatical notation (making it easier, typically, to correctly code mathmatical constructs)

vectorization results in more “Pythonic” code. Without vectorization, our code would be littered with inefficient and difficult to read for loops.

Broadcasting is the term used to describe the implicit element-by-element behavior of operations;

1.2 Data Types and Structure

np.array (homogenous multidimensional array. it is a table of elments,all of the same type, indexed by a tuple of non-negative integers)

note : numpy.array is not the same as the standard python library class array.array

dtype	变体	描述
int	int8, int16, int32, int64	整数类型
uint	uint8, uint16, uint32, uint64	无符号（非负）整型
bool	bool	布尔类型（True False）
float	float16, float32, float64, float128	浮点型
complex	complex64, complex128, complex256	复数浮点型

b1, i1, i2, i4, i8, u1, u2, u4, u8, f2, f4, f8, c8, c16, a<n> 或者是下面的也可以

int8,...,uint8,...,float16, float32, float64, complex64, complex128

1.3. Data Acquisition and Cleaning

I/O

np.genfromtxt(s, delimiter=",",dtype = np.int)

print()

One-dimensional array are then printed as rows, bidiemensional as matrices and tridimensional as lists of matrices

Create

You can create numpy array from a regular python list or tuple using array function, the types of the resulting array is deduced from the type of the elements in the sequences. A frequent error consists in calling array with multiple arguments, rather than providing a single sequence as an argument.

The elements of an array are originaly unknows, but its size is known. Numpy offers serveral functions to create arrays with initial placeholder content. These minimize the necessity of growing arrays, an expensive operation.

❤ To create sequences of numbers

np.arange, anlogous to the Pyhton built-in range return an array, rather than an list

random normal

Z = np.pad(Z, pad_width=1, mode='constant', constant_values=0) Extent the border of the array

Z = np.tile( np.array([[0,1],[1,0]]), (4,4))

np.fromiter(iterable, float)

iterable = (x*x for x in range(5))
np.fromiter(iterable, float)
array([  0.,   1.,   4.,   9.,  16.])

User-defines function

❤Structured Array

Numpy支持创建更加复杂的结构化数组，底层其实就是C中的结构体，每个元素可以包含不同类型的数据。

结构数组创建方式（dtype 创建方式）

利用字符串创建dtype类型

2. 使用字典创建dtype类型

定义数据类型(包括名字和格式)Tuple {‘names’: ('name', 'age', 'sex', 'weight'),'formats':('U10','i4','U6', 'f8') }

创建结构化数组，使用定义好的数据类型

students=np.array([('袁菲',25,'女',55),('张三',22,'女',65),('李四',28,'男',70),('赵二',21,'女',49),('王五',29,'男',85)],dtype=student_type)

3. 使用列表创建dtype类型

4. 使用元组创建dtype类型

❓

dtype类型相关操作

获取键值，获取列值

data['name']

data[['col1','col2','col3']]

data[data["grade"]>26]["name"]

获取对象值，行值

data[0]

Inspect

ndarry.ndim the number of axes of the array

ndarry.shape the dimensions of the arry

ndarray.size the total number of elements of the array. This is euqal to the product of the elements of shape

ndarray.dtype an object describing the type of the elementa in the array

a.dtype.name

ndarray.itemsize the size in bytes of each elment of the array

ndarray.data the buffer containg the actual elements of the array

1.4. Data Manipulation

1.4.1 Selction - SubSetting , Slicing , Indexing (索引而并非本体操作，一种次序属性)

💡

1. Slicing 时视图操作（引用原始内存中的数据，改变切片原数据也会改变），而Indexing并非视图操作（返回新的独立的数组）

Dimension

One-dimensional arrays

Multidimensiona Arrays

Slicing

Slicing Method	Explanation
a[m]	选择索引为 m 的元素，其中 m 为整数
a[-m]	从列表末尾开始选择第 m 个元素，其中m是整数
a[m: n]	选择索引从 m 开始到 n-1 结束的元素
a[:], a[0: -1]	选择给定维的所有元素
a[:n]	选择索引从 0 开始 n-1 结束的元素
a[m:], a[m: -1]	选择索引从 m 开始到数组最后的所有元素
a[m: n: p]	选择索引从 m 开始 n 结束，增量为 p 的所有元素
a[: : -1]	以逆序，选择所有元素
Matrix	ㅤ
a[-1] = a[-1,:]	The expression within brackets in b[i] is treated as an i followed by as many instance of as needed to represent the remaining axes
b[i,...]	x[1,2,...] is equivalent to x[1,2,:,:,:],
a[[1], :] = a[1:2, :]	ㅤ
a[::2, ::2]	提取指数为偶数的元素
a[1::2, 1::3]	提取指数为奇数的元素

Fancy Indexing

🔺 Integer array indexing: When you index into numpy arrays using slicing, the resulting array view will always be a subarray of the original array. In contrast, integer array indexing allows you to construct arbitrary arrays using the data from another array. Here is an example:

2. Boolean Array indexing: Boolean array indexing lets you pick out arbitrary elements of an array. Frequently this type of indexing is used to select the elements of an array that satisfy some condition.

3. ix_函数可用于组合不同的向量，以便获得每个n-uplet的结果.如果要计算从每个向量a，b和c中取得的所有三元组的所有a + b * c：

1.4.2 Reaction - Array Mathematics (Arithmetic Operations, Aggregate Functions)

Aggregate Function (axis=0 沿着列，axis =1沿着行) Unary Operation, 这个和 Pandas 的Dataframe数据类型操作相反

ufunc (Univeral function) These functions operate elementwise on an array producing an array as output

08-Aggregation-Operator-in-Numpy.ipynb

10.7KB

np.where()

三个参数 np.where(cond, x, y) ,满足条件输出x, 否则输出y

一个参数 np.where(array): 输出非0值坐标

❤Broadcasting

The term boardcasting describes how numpy treats array with different shapes durring arithmetic operations.

provide a menas of vectorizing array operations so that looping occurs in C instead of Python

without making needless copies of data and usually leads to efficient algorithm implementations

Scalar Boradcasting

The stretching analogy conceptual model ： We can think of the scalar b being streched during the arithmetic operation into an array with the same shape as a. the new elements in b are simply copies of the original sclar.

Real Constion: NumPy is smart enough to use the original scalar value without actually making copies so that broadcasting operations are as memory and computationally efficient as possible.

General Boradcasting Rules

维度相等

对应维度上存在一个为1

在1的维度上扩展，可以看成先扩展程目矩阵大小，然后点乘

✔

❌

Examples

np.argmax(A) # 最大值的索引位置

1.4.3 Search and Prospect - Searching , Comparison，Sorting, Filtering

np.nonzeros() find the indices of non-zeros elements in the array

print(np.unravel_index(99,(6,7,8)))

print(np.intersect1d(Z1,Z2))

1.4.4 Manipulation- Combine, Reshape, Splitting（本体操作）,Condition Calculation

Reshape

Combine(stack)

np.r_()

np.c_()

Splitting

Condition Calculation

1.4.5 Copy and View

b= a Simple assignments make no copy of array objects or of their data.

c.view() Different array objects can share the same data. The view method creates a new array object that looks at the same data.

d = a.copy() The copy method makes a complete copy of the array and its data.

1.5. Data Analysis and Visulization

Stage 2 — Projects and Exercises

Project 01 : 100 numpy exercises

This is a collection of exercises that have been collected in the numpy mailing list, on stack overflow and in the numpy documentation. The goal of this collection is to offer a quick reference for both old and new users but also to provide a set of exercises for those who teach.

If you find an error or think you've a better way to solve some of them, feel free to open an issue at https://github.com/rougier/numpy-100. File automatically generated. See the documentation to update questions/answers/hints programmatically.

1-20

1. Import the numpy package under the name `np` (★☆☆)

hint: import … as

python import numpy as np

2. Print the numpy version and the configuration (★☆☆)

hint: np.__version__, np.show_config)

python print(np.__version__) np.show_config()

3. Create a null vector of size 10 (★☆☆)

hint: np.zeros

python

Z = np.zeros(10)

print(Z)

4. How to find the memory size of any array (★☆☆)

hint: size, itemsize

python

Z = np.zeros((10,10))

print(Z.datatype.name)

print("%d bytes" % (Z.size * Z.itemsize))

5. How to get the documentation of the numpy add function from the command line? (★☆☆)

hint: np.info

python

import numpy;

numpy.info(numpy.add)

6. Create a null vector of size 10 but the fifth value which is 1 (★☆☆)

hint: array[4]

python

Z = np.zeros(10)

Z[4] = 1

print(Z)

7. Create a vector with values ranging from 10 to 49 (★☆☆)

hint: arange

python

Z = np.arange(10,50)

print(Z)

❤ 8. Reverse a vector (first element becomes last) (★☆☆)

hint: array[::-1]

python

Z = np.arange(50)

Z = Z[::-1]

print(Z)

9. Create a 3x3 matrix with values ranging from 0 to 8 (★☆☆)

hint: reshape

python

nz = np.nonzero([1,2,0,0,4,0])

print(nz)

❤10. Find indices of non-zero elements from [1,2,0,0,4,0] (★☆☆)

hint: np.nonzero

nz = np.nonzero([1,2,0,0,4,0])

print(nz)

11. Create a 3x3 identity matrix (★☆☆)

hint: np.eye

python

Z = np.eye(3)

print(Z)

12. Create a 3x3x3 array with random values (★☆☆)

hint: np.random.random

python Z = np.random.random((3,3,3)) print(Z)

13. Create a 10x10 array with random values and find the minimum and maximum values (★☆☆)

hint: min, max

Z = np.random.random((10,10))

Zmin, Zmax = Z.min(), Z.max()

print(Zmin, Zmax)

14. Create a random vector of size 30 and find the mean value (★☆☆)

hint: mean

python

Z = np.random.random(30)

m = Z.mean()

print(m)

15. Create a 2d array with 1 on the border and 0 inside (★☆☆)

hint: array[1:-1, 1:-1]

Z = np.ones((10,10))

Z[1:-1,1:-1] = 0

print(Z)

❤16. How to add a border (filled with 0's) around an existing array? (★☆☆)

hint: np.pad

Z = np.ones((5,5))

Z = np.pad(Z, pad_width=1, mode='constant', constant_values=0)

print(Z)

17. What is the result of the following expression? (★☆☆)

python

0 * np.nan

np.nan == np.nan

np.inf > np.nan

np.nan - np.nan

np.nan in set([np.nan])

❤0.3 == 3 * 0.1

hint: NaN = not a number, inf = infinity

print(0 * np.nan) Nan

print(np.nan == np.nan) False

print(np.inf > np.nan) False

print(np.nan - np.nan) nan

print(np.nan in set([np.nan])) True

print(0.3 == 3 * 0.1) False

❤18. Create a 5x5 matrix with values 1,2,3,4 just below the diagonal (★☆☆)

hint: np.diag

Z = np.diag(1+np.arange(4),k=-1)

print(Z)

❤19. Create a 8x8 matrix and fill it with a checkerboard pattern (★☆☆)

hint: array[::2]

Z = np.zeros((8,8),dtype=int)

Z[1::2,::2] = 1

Z[::2,1::2] = 1

print(Z)

❤20. Consider a (6,7,8) shape array, what is the index (x,y,z) of the 100th element?

hint: np.unravel_index

python

print(np.unravel_index(99,(6,7,8)))

21-40

❤21. Create a checkerboard 8x8 matrix using the tile function (★☆☆)

hint: np.tile

Z = np.tile( np.array([[0,1],[1,0]]), (4,4))

print(Z)

22. Normalize a 5x5 random matrix (★☆☆)

hint: (x -mean)/std

Z = np.random.random((5,5))

Z = (Z - np.mean (Z)) / (np.std (Z))

print(Z)

23. Create a custom dtype that describes a color as four unsigned bytes (RGBA) (★☆☆)

hint: np.dtype

color = np.dtype([("r", np.ubyte, 1), ("g", np.ubyte, 1), ("b", np.ubyte, 1), ("a", np.ubyte, 1)])

24. Multiply a 5x3 matrix by a 3x2 matrix (real matrix product) (★☆☆)

hint:

Z = np.dot(np.ones((5,3)), np.ones((3,2))

Z = np.ones((5,3)) @ np.ones((3,2)) print(Z)

❤25. Given a 1D array, negate all elements which are between 3 and 8, in place. (★☆☆)

hint: >, <=

Z = np.arange(11)

Z[(3 < Z) & (Z <= 8)] *= -1

print(Z)

26. What is the output of the following script? (★☆☆)

print(sum(range(5),-1)) from numpy import * print(sum(range(5),-1)) ``` hint: np.sum

27. Consider an integer vector Z, which of these expressions are legal? (★☆☆)

Z**Z

2 << Z >> 2

Z <- Z

1j*Z

Z/1/1

Z<Z>Z

28. What are the result of the following expressions?

python

np.array(0) / np.array(0)

np.array(0) // np.array(0)

np.array([np.nan]).astype(int).astype(float)

❤29. How to round away from zero a float array ? (★☆☆)

hint: np.uniform, np.copysign, np.ceil, np.abs

Z = np.random.uniform(-10,+10,10)

print (np.copysign(np.ceil(np.abs(Z)), Z))

❤30. How to find common values between two arrays? (★☆☆)

hint: np.intersect1d

Z1 = np.random.randint(0,10,10)

Z2 = np.random.randint(0,10,10)

print(np.intersect1d(Z1,Z2))

31. How to ignore all numpy warnings (not recommended)? (★☆☆)

hint: np.seterr, np.errstate

defaults = np.seterr(all="ignore")

Z = np.ones(1) / 0

_ = np.seterr(**defaults)

wqwqw;l-

32. Is the following expressions true? (★☆☆)

python np.sqrt(-1) == np.emath.sqrt(-1)

hint: imaginary number

python

❤np.sqrt(-1) == np.emath.sqrt(-1)

❤33. How to get the dates of yesterday, today and tomorrow? (★☆☆)

hint: np.datetime64, np.timedelta64

python

yesterday = np.datetime64('today', 'D') - np.timedelta64(1, 'D')

today = np.datetime64('today', 'D')

tomorrow = np.datetime64('today', 'D') + np.timedelta64(1, 'D')

34. How to get all the dates corresponding to the month of July 2016? (★★☆)

hint: np.arange(dtype=datetime64['D'])

python

Z = np.arange('2016-07', '2016-08', dtype='datetime64[D]') print(Z)

**35. How to compute ((A+B)*(-A/2)) in place (without copy)? (★★☆)**

hint: np.add(out=), np.negative(out=), np.multiply(out=), np.divide(out=)

A = np.ones(3)*1

B = np.ones(3)*2

C = np.ones(3)*3

np.add(A,B,out=B)

np.divide(A,2,out=A)

np.negative(A,out=A)

np.multiply(A,B,out=A)

❤36. Extract the integer part of a random array using 5 different methods (★★☆)

hint: %, np.floor, np.ceil, astype, np.trunc

Z = np.random.uniform(0,10,10)

print (Z - Z%1)

print (np.floor(Z))

print (np.ceil(Z)-1)

print (Z.astype(int))

print (np.trunc(Z))

❤37. Create a 5x5 matrix with row values ranging from 0 to 4 (★★☆)

hint: np.arange

Z = np.zeros((5,5))

Z += np.arange(5)

print(Z)

38. Consider a generator function that generates 10 integers and use it to build an array (★☆☆)

hint: np.fromiter

python

def generate():

for x in range(10):

yield x

Z = np.fromiter(generate(),dtype=float,count=-1)

print(Z)

39. Create a vector of size 10 with values ranging from 0 to 1, both excluded (★★☆)

hint: np.linspace

Z = np.linspace(0,1,11,endpoint=False)[1:]

print(Z)

40. Create a random vector of size 10 and sort it (★★☆)

hint: sort

Z = np.random.random(10)

Z.sort()

print(Z)

41-60

41. How to sum a small array faster than np.sum? (★★☆)

hint: np.add.reduce

42. Consider two random array A and B, check if they are equal (★★☆)

hint: np.allclose, np.array_equal

43. Make an array immutable (read-only) (★★☆)

hint: flags.writeable

44. Consider a random 10x2 matrix representing cartesian coordinates, convert them to polar coordinates (★★☆)

hint: np.sqrt, np.arctan2

45. Create random vector of size 10 and replace the maximum value by 0 (★★☆)

hint: argmax

46. Create a structured array with `x` and `y` coordinates covering the [0,1]x[0,1] area (★★☆)

hint: np.meshgrid

47. Given two arrays, X and Y, construct the Cauchy matrix C (Cij =1/(xi - yj))

hint: np.subtract.outer

48. Print the minimum and maximum representable value for each numpy scalar type (★★☆)

hint: np.iinfo, np.finfo, eps

49. How to print all the values of an array? (★★☆)

hint: np.set_printoptions

50. How to find the closest value (to a given scalar) in a vector? (★★☆)

hint: argmin

51. Create a structured array representing a position (x,y) and a color (r,g,b) (★★☆)

hint: dtype

❓52. Consider a random vector with shape (100,2) representing coordinates, find point by point distances (★★☆)

hint: np.atleast_2d, T, np.sqrt

53. How to convert a float (32 bits) array into an integer (32 bits) in place?

hint: view and [:] =

❤54. How to read the following file? (★★☆)

hint: np.genfromtxt

55. What is the equivalent of enumerate for numpy arrays? (★★☆)

hint: np.ndenumerate, np.ndindex

56. Generate a generic 2D Gaussian-like array (★★☆)

hint: np.meshgrid, np.exp

57. How to randomly place p elements in a 2D array? (★★☆)

hint: np.put, np.random.choice

58. Subtract the mean of each row of a matrix (★★☆)

hint: mean(axis=,keepdims=)

59. How to sort an array by the nth column? (★★☆)

hint: argsort

60. How to tell if a given 2D array has null columns? (★★☆)

hint: any, ~

61-80

61. Find the nearest value from a given value in an array (★★☆)

hint: np.abs, argmin, flat

python Z = np.random.uniform(0,1,10) z = 0.5 m = Z.flat[np.abs(Z - z).argmin()] print(m)

62. Considering two arrays with shape (1,3) and (3,1), how to compute their sum using an iterator? (★★☆)

hint: np.nditer

python A = np.arange(3).reshape(3,1) B = np.arange(3).reshape(1,3) it = np.nditer([A,B,None]) for x,y,z in it: z[...] = x + y print(it.operands[2])

63. Create an array class that has a name attribute (★★☆)

hint: class method

```python class NamedArray(np.ndarray): def new(cls, array, name="no name"): obj = np.asarray(array).view(cls) obj.name = name return obj def _arrayfinalize(self, obj): if obj is None: return self.info = getattr(obj, 'name', "no name")

Z = NamedArray(np.arange(10), "range_10") print (Z.name) ```

64. Consider a given vector, how to add 1 to each element indexed by a second vector (be careful with repeated indices)? (★★★)

hint: np.bincount | np.add.at

```python

Author: Brett Olsen

Z = np.ones(10) I = np.random.randint(0,len(Z),20) Z += np.bincount(I, minlength=len(Z)) print(Z)

Another solution

Author: Bartosz Telenczuk

np.add.at(Z, I, 1) print(Z) ```

65. How to accumulate elements of a vector (X) to an array (F) based on an index list (I)? (★★★)

hint: np.bincount

```python

Author: Alan G Isaac

X = [1,2,3,4,5,6] I = [1,3,9,3,4,1] F = np.bincount(I,X) print(F) ```

66. Considering a (w,h,3) image of (dtype=ubyte), compute the number of unique colors (★★★)

hint: np.unique

```python

Author: Nadav Horesh

w,h = 16,16 I = np.random.randint(0,2,(h,w,3)).astype(np.ubyte) F = I[...,0]256256 + I[...,1]*256 +I[...,2] n = len(np.unique(F)) print(np.unique(I)) ```

67. Considering a four dimensions array, how to get sum over the last two axis at once? (★★★)

hint: sum(axis=(-2,-1))

```python A = np.random.randint(0,10,(3,4,3,4))

solution by passing a tuple of axes (introduced in numpy 1.7.0)

sum = A.sum(axis=(-2,-1)) print(sum)

solution by flattening the last two dimensions into one

(useful for functions that don't accept tuples for axis argument)

sum = A.reshape(A.shape[:-2] + (-1,)).sum(axis=-1) print(sum) ```

68. Considering a one-dimensional vector D, how to compute means of subsets of D using a vector S of same size describing subset indices? (★★★)

hint: np.bincount

```python

Author: Jaime Fernández del Río

D = np.random.uniform(0,1,100) S = np.random.randint(0,10,100) Dsums = np.bincount(S, weights=D) Dcounts = np.bincount(S) Dmeans = Dsums / Dcounts print(Dmeans)

Pandas solution as a reference due to more intuitive code

import pandas as pd print(pd.Series(D).groupby(S).mean()) ```

69. How to get the diagonal of a dot product? (★★★)

hint: np.diag

```python

Author: Mathieu Blondel

A = np.random.uniform(0,1,(5,5)) B = np.random.uniform(0,1,(5,5))

Slow version

np.diag(np.dot(A, B))

Fast version

np.sum(A * B.T, axis=1)

Faster version

np.einsum("ij,ji->i", A, B) ```

70. Consider the vector [1, 2, 3, 4, 5], how to build a new vector with 3 consecutive zeros interleaved between each value? (★★★)

hint: array[::4]

```python

Author: Warren Weckesser

Z = np.array([1,2,3,4,5]) nz = 3 Z0 = np.zeros(len(Z) + (len(Z)-1)*(nz)) Z0[::nz+1] = Z print(Z0) ```

71. Consider an array of dimension (5,5,3), how to mulitply it by an array with dimensions (5,5)? (★★★)

hint: array[:, :, None]

python A = np.ones((5,5,3)) B = 2*np.ones((5,5)) print(A * B[:,:,None])

72. How to swap two rows of an array? (★★★)

hint: array[[]] = array[[]]

```python

Author: Eelco Hoogendoorn

A = np.arange(25).reshape(5,5) A[[0,1]] = A[[1,0]] print(A) ```

73. Consider a set of 10 triplets describing 10 triangles (with shared vertices), find the set of unique line segments composing all the triangles (★★★)

hint: repeat, np.roll, np.sort, view, np.unique

```python

Author: Nicolas P. Rougier

faces = np.random.randint(0,100,(10,3)) F = np.roll(faces.repeat(2,axis=1),-1,axis=1) F = F.reshape(len(F)*3,2) F = np.sort(F,axis=1) G = F.view( dtype=[('p0',F.dtype),('p1',F.dtype)] ) G = np.unique(G) print(G) ```

74. Given an array C that is a bincount, how to produce an array A such that np.bincount(A) == C? (★★★)

hint: np.repeat

```python

Author: Jaime Fernández del Río

C = np.bincount([1,1,2,3,4,4,6]) A = np.repeat(np.arange(len(C)), C) print(A) ```

75. How to compute averages using a sliding window over an array? (★★★)

hint: np.cumsum

```python

Author: Jaime Fernández del Río

def movingaverage(a, n=3) : ret = np.cumsum(a, dtype=float) ret[n:] = ret[n:] - ret[:-n] return ret[n - 1:] / n Z = np.arange(20) print(movingaverage(Z, n=3)) ```

76. Consider a one-dimensional array Z, build a two-dimensional array whose first row is (Z[0],Z[1],Z[2]) and each subsequent row is shifted by 1 (last row should be (Z[-3],Z[-2],Z[-1]) (★★★)

hint: from numpy.lib import stride_tricks

```python

Author: Joe Kington / Erik Rigtorp

from numpy.lib import stride_tricks

def rolling(a, window): shape = (a.size - window + 1, window) strides = (a.itemsize, a.itemsize) return stridetricks.asstrided(a, shape=shape, strides=strides) Z = rolling(np.arange(10), 3) print(Z) ```

77. How to negate a boolean, or to change the sign of a float inplace? (★★★)

hint: np.logical_not, np.negative

```python

Author: Nathaniel J. Smith

Z = np.random.randint(0,2,100) np.logical_not(Z, out=Z)

Z = np.random.uniform(-1.0,1.0,100) np.negative(Z, out=Z) ```

78. Consider 2 sets of points P0,P1 describing lines (2d) and a point p, how to compute distance from p to each line i (P0[i],P1[i])? (★★★)

No hints provided...

```python def distance(P0, P1, p): T = P1 - P0 L = (T2).sum(axis=1) U = -((P0[:,0]-p[...,0])T[:,0] + (P0[:,1]-p[...,1])T[:,1]) / L U = U.reshape(len(U),1) D = P0 + UT - p return np.sqrt((D*2).sum(axis=1))

P0 = np.random.uniform(-10,10,(10,2)) P1 = np.random.uniform(-10,10,(10,2)) p = np.random.uniform(-10,10,( 1,2)) print(distance(P0, P1, p)) ```

79. Consider 2 sets of points P0,P1 describing lines (2d) and a set of points P, how to compute distance from each point j (P[j]) to each line i (P0[i],P1[i])? (★★★)

No hints provided...

```python

Author: Italmassov Kuanysh

based on distance function from previous question

P0 = np.random.uniform(-10, 10, (10,2)) P1 = np.random.uniform(-10,10,(10,2)) p = np.random.uniform(-10, 10, (10,2)) print(np.array([distance(P0,P1,pi) for pi in p])) ```

80. Consider an arbitrary array, write a function that extract a subpart with a fixed shape and centered on a given element (pad with a `fill` value when necessary) (★★★)

hint: minimum maximum

```python

Author: Nicolas Rougier

Z = np.random.randint(0,10,(10,10)) shape = (5,5) fill = 0 position = (1,1)

R = np.ones(shape, dtype=Z.dtype)*fill P = np.array(list(position)).astype(int) Rs = np.array(list(R.shape)).astype(int) Zs = np.array(list(Z.shape)).astype(int)

Rstart = np.zeros((len(shape),)).astype(int) Rstop = np.array(list(shape)).astype(int) Zstart = (P-Rs//2) Zstop = (P+Rs//2)+Rs%2

Rstart = (Rstart - np.minimum(Zstart,0)).tolist() Zstart = (np.maximum(Zstart,0)).tolist() Rstop = np.maximum(Rstart, (Rstop - np.maximum(Z_stop-Zs,0))).tolist() Zstop = (np.minimum(Zstop,Zs)).tolist()

r = [slice(start,stop) for start,stop in zip(Rstart,Rstop)] z = [slice(start,stop) for start,stop in zip(Zstart,Zstop)] R[r] = Z[z] print(Z) print(R) ```

81-100

81. Consider an array Z = [1,2,3,4,5,6,7,8,9,10,11,12,13,14], how to generate an array R = [[1,2,3,4], [2,3,4,5], [3,4,5,6], ..., [11,12,13,14]]? (★★★)

hint: stride_tricks.as_strided

```python

Author: Stefan van der Walt

Z = np.arange(1,15,dtype=np.uint32) R = stridetricks.asstrided(Z,(11,4),(4,4)) print(R) ```

82. Compute a matrix rank (★★★)

hint: np.linalg.svd

```python

Author: Stefan van der Walt

Z = np.random.uniform(0,1,(10,10)) U, S, V = np.linalg.svd(Z) # Singular Value Decomposition rank = np.sum(S > 1e-10) print(rank) ```

83. How to find the most frequent value in an array?

hint: np.bincount, argmax

python Z = np.random.randint(0,10,50) print(np.bincount(Z).argmax())

84. Extract all the contiguous 3x3 blocks from a random 10x10 matrix (★★★)

hint: stride_tricks.as_strided

```python

Author: Chris Barker

Z = np.random.randint(0,5,(10,10)) n = 3 i = 1 + (Z.shape[0]-3) j = 1 + (Z.shape[1]-3) C = stridetricks.asstrided(Z, shape=(i, j, n, n), strides=Z.strides + Z.strides) print(C) ```

85. Create a 2D array subclass such that Z[i,j] == Z[j,i] (★★★)

hint: class method

```python

Author: Eric O. Lebigot

Note: only works for 2d array and value setting using indices

class Symetric(np.ndarray): def setitem(self, index, value): i,j = index super(Symetric, self).setitem((i,j), value) super(Symetric, self).setitem((j,i), value)

def symetric(Z): return np.asarray(Z + Z.T - np.diag(Z.diagonal())).view(Symetric)

S = symetric(np.random.randint(0,10,(5,5))) S[2,3] = 42 print(S) ```

86. Consider a set of p matrices wich shape (n,n) and a set of p vectors with shape (n,1). How to compute the sum of of the p matrix products at once? (result has shape (n,1)) (★★★)

hint: np.tensordot

```python

Author: Stefan van der Walt

p, n = 10, 20 M = np.ones((p,n,n)) V = np.ones((p,n,1)) S = np.tensordot(M, V, axes=[[0, 2], [0, 1]]) print(S)

It works, because:

M is (p,n,n)

V is (p,n,1)

Thus, summing over the paired axes 0 and 0 (of M and V independently),

and 2 and 1, to remain with a (n,1) vector.

```

87. Consider a 16x16 array, how to get the block-sum (block size is 4x4)? (★★★)

hint: np.add.reduceat

```python

Author: Robert Kern

Z = np.ones((16,16)) k = 4 S = np.add.reduceat(np.add.reduceat(Z, np.arange(0, Z.shape[0], k), axis=0), np.arange(0, Z.shape[1], k), axis=1) print(S) ```

88. How to implement the Game of Life using numpy arrays? (★★★)

No hints provided...

```python

Author: Nicolas Rougier

def iterate(Z): # Count neighbours N = (Z[0:-2,0:-2] + Z[0:-2,1:-1] + Z[0:-2,2:] + Z[1:-1,0:-2] + Z[1:-1,2:] + Z[2: ,0:-2] + Z[2: ,1:-1] + Z[2: ,2:])

Z = np.random.randint(0,2,(50,50)) for i in range(100): Z = iterate(Z) print(Z) ```

89. How to get the n largest values of an array (★★★)

hint: np.argsort | np.argpartition

```python Z = np.arange(10000) np.random.shuffle(Z) n = 5

Slow

print (Z[np.argsort(Z)[-n:]])

Fast

print (Z[np.argpartition(-Z,n)[:n]]) ```

90. Given an arbitrary number of vectors, build the cartesian product (every combinations of every item) (★★★)

hint: np.indices

```python

Author: Stefan Van der Walt

def cartesian(arrays): arrays = [np.asarray(a) for a in arrays] shape = (len(x) for x in arrays)

print (cartesian(([1, 2, 3], [4, 5], [6, 7]))) ```

91. How to create a record array from a regular array? (★★★)

hint: np.core.records.fromarrays

python Z = np.array([("Hello", 2.5, 3), ("World", 3.6, 2)]) R = np.core.records.fromarrays(Z.T, names='col1, col2, col3', formats = 'S8, f8, i8') print(R)

92. Consider a large vector Z, compute Z to the power of 3 using 3 different methods (★★★)

hint: np.power, *, np.einsum

```python

Author: Ryan G.

x = np.random.rand(int(5e7))

%timeit np.power(x,3) %timeit xxx %timeit np.einsum('i,i,i->i',x,x,x) ```

93. Consider two arrays A and B of shape (8,3) and (2,2). How to find rows of A that contain elements of each row of B regardless of the order of the elements in B? (★★★)

hint: np.where

```python

Author: Gabe Schwartz

A = np.random.randint(0,5,(8,3)) B = np.random.randint(0,5,(2,2))

C = (A[..., np.newaxis, np.newaxis] == B) rows = np.where(C.any((3,1)).all(1))[0] print(rows) ```

94. Considering a 10x3 matrix, extract rows with unequal values (e.g. [2,2,3]) (★★★)

No hints provided...

```python

Author: Robert Kern

Z = np.random.randint(0,5,(10,3)) print(Z)

solution for arrays of all dtypes (including string arrays and record arrays)

E = np.all(Z[:,1:] == Z[:,:-1], axis=1) U = Z[~E] print(U)

soluiton for numerical arrays only, will work for any number of columns in Z

U = Z[Z.max(axis=1) != Z.min(axis=1),:] print(U) ```

95. Convert a vector of ints into a matrix binary representation (★★★)

hint: np.unpackbits

```python

Author: Warren Weckesser

I = np.array([0, 1, 2, 3, 15, 16, 32, 64, 128]) B = ((I.reshape(-1,1) & (2**np.arange(8))) != 0).astype(int) print(B[:,::-1])

Author: Daniel T. McDonald

I = np.array([0, 1, 2, 3, 15, 16, 32, 64, 128], dtype=np.uint8) print(np.unpackbits(I[:, np.newaxis], axis=1)) ```

96. Given a two dimensional array, how to extract unique rows? (★★★)

hint: np.ascontiguousarray | np.unique

```python

Author: Jaime Fernández del Río

Z = np.random.randint(0,2,(6,3)) T = np.ascontiguousarray(Z).view(np.dtype((np.void, Z.dtype.itemsize * Z.shape[1]))) , idx = np.unique(T, returnindex=True) uZ = Z[idx] print(uZ)

Author: Andreas Kouzelis

NumPy >= 1.13

uZ = np.unique(Z, axis=0) print(uZ) ```

97. Considering 2 vectors A & B, write the einsum equivalent of inner, outer, sum, and mul function (★★★)

hint: np.einsum

```python

Author: Alex Riley

Make sure to read: http://ajcr.net/Basic-guide-to-einsum/

A = np.random.uniform(0,1,10) B = np.random.uniform(0,1,10)

np.einsum('i->', A) # np.sum(A) np.einsum('i,i->i', A, B) # A * B np.einsum('i,i', A, B) # np.inner(A, B) np.einsum('i,j->ij', A, B) # np.outer(A, B) ```

98. Considering a path described by two vectors (X,Y), how to sample it using equidistant samples (★★★)?

hint: np.cumsum, np.interp

```python

Author: Bas Swinckels

phi = np.arange(0, 10np.pi, 0.1) a = 1 x = aphi*np.cos(phi) y = aphinp.sin(phi)

dr = (np.diff(x)2 + np.diff(y)2)**.5 # segment lengths r = np.zeroslike(x) r[1:] = np.cumsum(dr) # integrate path rint = np.linspace(0, r.max(), 200) # regular spaced path xint = np.interp(rint, r, x) # integrate path yint = np.interp(rint, r, y) ```

99. Given an integer n and a 2D array X, select from X the rows which can be interpreted as draws from a multinomial distribution with n degrees, i.e., the rows which only contain integers and which sum to n. (★★★)

hint: np.logical_and.reduce, np.mod

```python

Author: Evgeni Burovski

X = np.asarray([[1.0, 0.0, 3.0, 8.0], [2.0, 0.0, 1.0, 1.0], [1.5, 2.5, 1.0, 0.0]]) n = 4 M = np.logical_and.reduce(np.mod(X, 1) == 0, axis=-1) M &= (X.sum(axis=-1) == n) print(X[M]) ```

100. Compute bootstrapped 95% confidence intervals for the mean of a 1D array X (i.e., resample the elements of an array with replacement N times, compute the mean of each sample, and then compute percentiles over the means). (★★★)

hint: np.percentile

```python

Author: Jessica B. Hamrick

X = np.random.randn(100) # random 1D array N = 1000 # number of bootstrap samples idx = np.random.randint(0, X.size, (N, X.size)) means = X[idx].mean(axis=1) confint = np.percentile(means, [2.5, 97.5]) print(confint) ```

Project 02 : FFT& Wavelets

构建分段函数

方法1： np.piecewise(x, condlist, funclist)

方法2：构建IND指针

方法3：构建条件

error：The truth value of an array with more than one element is ambiguous

位运算

运算符优先级

遍历函数 enumerate(sequence, [start=0])

将一个可遍历的数据对象(如列表、元组或字符串)组合为一个索引序列，同时列出数据和数据下标，一般用在 for 循环当中。

Numpy 数据类型和类型转换

数据类型设置

转换数据类型

python 积分

定积分

integrate(函数，（变量，下限，上限）)

函数积分

数值积分

Project 03 : fit and interpolation predict

插值，对准了才可以插吗，那就一定得过数据点。拟合，就是要得到最接近的结果，是要看总体效果。

插值类别

Lagrange插值Lagrange插值是n次多项式插值，其成功地用构造插值基函数的方法解决了求n次多项式插值函数问题。

★基本思想将待求的n次多项式插值函数pn(x）改写成另一种表示方式，再利用插值条件⑴确定其中的待定函数，从而求出插值多项式。

Newton插值Newton插值也是n次多项式插值，它提出另一种构造插值多项式的方法，与Lagrange插值相比，具有承袭性和易于变动节点的特点。

★基本思想将待求的n次插值多项式Pn（x）改写为具有承袭性的形式，然后利用插值条件⑴确定Pn（x）的待定系数，以求出所要的插值函数。

Hermite插值Hermite插值是利用未知函数f(x)在插值节点上的函数值及导数值来构造插值多项式的，其提法为：给定n+1个互异的节点x0,x1，……,xn上的函数值和导数值求一个2n+1次多项式H2n+1(x)满足插值条件H2n+1(xk)=ykH'2n+1(xk)=y'k k=0,1,2，……，n ⒀如上求出的H2n+1(x）称为2n+1次Hermite插值函数，它与被插函数一般有更好的密合度.

★基本思想利用Lagrange插值函数的构造方法，先设定函数形式，再利用插值条件⒀求出插值函数.

Numpy 拟合

线性拟合

二次拟合

scipy.optimize.curve_fit 拟合

对数，指数，幂数

高斯拟合

神经网络拟合

BP神经网络

Tensor Flow

插值

1.最小二乘拟合

实例1

实例2

2. 插值

FAQ

Numpy Data analysis practive sheet

数值 Python: 向量、矩阵和多维数组 | GreatX's Blog

rougier/numpy-100: 100 numpy exercises (with solutions)

Tutorials for Stanford cs228 and cs231n

Python 教程

Python常用库之Pandas

Author:felixfixit
URL:http://www.felixmicrospace.top/article/python_numpy
Copyright:All articles in this blog, except for special statements, adopt BY-NC-SA agreement. Please indicate the source!

Relate Posts

开发框架07- Doxygen 文档生成