Pandas Series and Dataframe

What is Series?

In pandas, a series is a one-dimensional labeled array capable of holding any data type. It is a fundamental data structure in pandas and can be thought of as a column in a spreadsheet or a single column of a DataFrame. The Series is similar to a NumPy array but comes with additional features like labeled indices, which makes it more powerful and flexible.

import pandas as pd

# Creating a Series with custom indices data = [1, 2, 3, 4, 5] custom_indices = ['a', 'b', 'c', 'd', 'e'] series = pd.Series(data, index=custom_indices)

#1st argument is Data, next is index # Displaying the Series with custom indices print(series)

O/P:

a 1 b 2 c 3 d 4 e 5 dtype: int64

How to access value from Series?

print(series['c'])

O/p:

3

What is DataFrame?

In pandas, a DataFrame is a two-dimensional labeled data structure that is widely used for handling and manipulating tabular data. It can be thought of as a table, similar to a spreadsheet or a SQL table, where data is organized in rows and columns. 

import pandas as pd
import numpy as np
from numpy.random import randn

np.random.seed(121)
# Create Dataframe
df = pd.DataFrame(randn(5, 4), ['A', 'B', 'C', 'D', 'E'], ['W', 'X', 'Y', 'Z'])
print("DataFrame:\n")
print(df)
# Access Column: Return type -> Series
print("Access Column named 'W'\n")
print(df['W'])
# Access list of Column : Return type -> Dataframe
print("Access Columns named Y & Z\n")
print(df[['Y', 'Z']])
# Create a new column
df['V'] = df['Y'] + df['Z']
print("DataFrame:\n")
print(df)
# Drop the new column, Inplace -> To drop row, use axis=0
df.drop('V', axis=1, inplace=True)
print("DataFrame:\n")
print(df)
# Access Row using its label : Return type -> Series
print("Access Row labeled as 'B':\n")
print(df.loc['B'])
# Access Row using its internal index : Return type -> Series
print("Access Row 1:\n")
print(df.iloc[1])
#Access multiple rows : Return type-> DataFrame
print("Access Row labeled as B & C:\n")
print(df.loc[['B','C']])

O/p:
DataFrame:

          W         X         Y         Z
A -0.212033 -0.284929 -0.573898 -0.440310
B -0.330111  1.183695  1.615373  0.367062
C -0.014119  0.629642  1.709641 -1.326987
D  0.401873 -0.191427  1.403826 -1.968769
E -0.790415 -0.732722  0.087744 -0.500286
Access Column named 'W'

A   -0.212033
B   -0.330111
C   -0.014119
D    0.401873
E   -0.790415
Name: W, dtype: float64
Access Columns named Y & Z

          Y         Z
A -0.573898 -0.440310
B  1.615373  0.367062
C  1.709641 -1.326987
D  1.403826 -1.968769
E  0.087744 -0.500286
DataFrame:

          W         X         Y         Z         V
A -0.212033 -0.284929 -0.573898 -0.440310 -1.014208
B -0.330111  1.183695  1.615373  0.367062  1.982435
C -0.014119  0.629642  1.709641 -1.326987  0.382653
D  0.401873 -0.191427  1.403826 -1.968769 -0.564943
E -0.790415 -0.732722  0.087744 -0.500286 -0.412542
DataFrame:

          W         X         Y         Z
A -0.212033 -0.284929 -0.573898 -0.440310
B -0.330111  1.183695  1.615373  0.367062
C -0.014119  0.629642  1.709641 -1.326987
D  0.401873 -0.191427  1.403826 -1.968769
E -0.790415 -0.732722  0.087744 -0.500286
Access Row labeled as 'B':

W   -0.330111
X    1.183695
Y    1.615373
Z    0.367062
Name: B, dtype: float64
Access Row 1:

W   -0.330111
X    1.183695
Y    1.615373
Z    0.367062
Name: B, dtype: float64
Access Row labeled as B & C:

          W         X         Y         Z
B -0.330111  1.183695  1.615373  0.367062
C -0.014119  0.629642  1.709641 -1.326987

Similarly,

# Access a particular cell
print(df.loc['B', 'Y'])
# Access a subset of DF
print(df.loc[['B','C'],['X','Y']])

1.6153729283493805

          X         Y
B  1.183695  1.615373
C  0.629642  1.709641

Cool features of Numpy

a = np.array([6,0,9,0,0,5,0])
def nonzero(a):
    return a!=0
print(a[nonzero(a)])

output:

[6 9 5]

 

Numpy Array vs Python List

 import numpy as np

list = [1,2,'Heaven','Hell']
arr = np.array(list)

#Please note, generally numpy generally does not contain Strings. It is mainly used

for numerical Operration.

list[2:] = 3

This is not allowed for normal python list. This will throw an error like

below:

    list[2:] = 3
TypeError: can only assign an iterable

However, it is allowed for numpy array and it will change the

original numpy array.

import numpy as np
list = [1,2,'Heaven','Hell']
arr = np.array(list)
arr[2:] = 3
print(arr)

Ouput: ['1' '2' '3' '3']

Now, no avoid changing the original array, we can use copy option as below.

Without copy:
import numpy as np
list = [1,2,'Heaven','Hell']
arr = np.array(list)
arr_copy =arr
arr_copy[2:] = 3
print(arr)
print(arr_copy)

O/p:
['1' '2' '3' '3']
['1' '2' '3' '3']

With copy:
import numpy as np
list = [1,2,'Heaven','Hell']
arr = np.array(list)
arr_copy =arr.copy()
arr_copy[2:] = 3
print(arr)
print(arr_copy)
O/p:
['1' '2' 'Heaven' 'Hell']
['1' '2' '3' '3']


list_one = [1,2,3]
list_two = [4,5,6]
print(f"Result of list addition: {list_one + list_two}")
arr_one = np.array(list_one)
arr_two = np.array(list_two)
print(f"Result of array addition: {arr_one + arr_two}")

O/p:
Result of list addition: [1, 2, 3, 4, 5, 6]
Result of array addition: [5 7 9]

Lamda Function

A lambda function is a small, anonymous function defined using the lambda keyword.

passwords = ['72tK3', '6zW5P3', '1P3=it3?', 'i"u7/auX{m']


allowed_passwords = list(filter(lambda x: len(x) >= 8, passwords))
print(allowed_passwords)

Filter Function

 Takes input a function which returns True/False and keeps the items which return True.

passwords = ['72tK3', '6zW5P3', '1P3=it3?', 'i"u7/auX{m']


def strong_password(password):
    return len(password) >= 8


allowed_passwords = list(filter(strong_password, passwords))
print(allowed_passwords)

Output:
['1P3=it3?', 'i"u7/auX{m']

Map function

 def names(firstname, lastname):

    """returns full name, inputs: first name and last name"""
    return firstname + ' ' + lastname

fname = 'Priyanka'
lname = 'Chakraborti'
name = names(fname,lname)
print(name)

Output:

Priyanka Chakraborti

# I want to apply the same function over list

list_of_firstnames = ['Andy', 'Sam', 'Jim', 'Micheal']
list_of_surnames = ['Smith', 'Doe', 'Baker', 'Hendrix']

result = map(names,list_of_firstnames,list_of_surnames)
list_of_names = list(result)
print(list_of_names)
Output:
['Andy Smith', 'Sam Doe', 'Jim Baker', 'Micheal Hendrix']

How to remove duplicates from list?

 Use set().

list_of_subjects= ['Physics', 'English', 'Statistics', 'English','Computer Science']

print(list(set(list_of_subjects)))

Output:

['Computer Science', 'English', 'Physics', 'Statistics']