16. Input/Output (IO)

IO operations are those dealing with interacting with files; either we are reading (input) or writing (output). Most of the IO operations are done with context managers to manage the lifecycle of acquiring and releasing access to the files.

16.1. File write and read

Here’s how to write to a file. Note that open() is a context manager for the file. Since we need to access the file in write mode, we pass in w.

1with open('data.csv', 'w') as f:
2    for r in range(100):
3        s = ','.join([str(c) for c in range(100)])
4        f.write(s)
5        f.write('\n')

Here’s how to read from a file. Since we need to access the file in read mode, we pass in r.

1with open('data.csv', 'r') as f:
2    for line in f:
3        print(line)

16.2. JSON file write and read

Creating and reading JSON file is pretty easy with the help of the json module. Note that we use the open() context manager as before, however, we leverage the dumps() method to dump dictionary data. The values stored in the dictionary should be primitive types (string, integer, float and boolean) or collections (list and dictionary) and the keys should be strings.

 1import json
 2
 3data = {f'x{x}': {f'y{y}': y for y in range(100)} for x in range(100)}
 4
 5# write json
 6with open('data.json', 'w') as f:
 7    f.write(json.dumps(data))
 8
 9# write pretty json
10with open('data.json', 'w') as f:
11    f.write(json.dumps(data, indent=2))
12
13# read json
14with open('data.json', 'r') as f:
15    s = json.load(f)
16    print(s)

16.3. Pickle/unpickle data

We have shown how to write and read from text-based files. Here, we show how to save data in binary format using the pickle module. Note that since the data being written and read is binary, we specify wb and rb, respectively.

 1import pickle
 2
 3data = {f'x{x}': {f'y{y}': y for y in range(100)} for x in range(100)}
 4
 5# pickle data
 6with open('data.p', 'wb') as f:
 7    pickle.dump(data, f)
 8
 9# unpickle data
10with open('data.p', 'rb') as f:
11    data = pickle.load(f)
12    print(data)

16.4. Shelving/unshelfing data

We can use the shelve module to save data as well. If you have logical sets of data and you do not want to create one pickle file per data, then you can shelve the datasets into one file.

 1import shelve
 2
 3data = {f'x{x}': {f'y{y}': y for y in range(100)} for x in range(100)}
 4
 5# shelving data
 6with shelve.open('data.s') as s:
 7    s['data'] = data
 8
 9# unshelving data
10with shelve.open('data.s') as s:
11    data = s['data']
12    print(data)

16.5. Creating temporary file and directory

If we need to create temporary files, we can use the tempfile module.

 1import tempfile
 2
 3# write to a temporary file
 4with tempfile.NamedTemporaryFile(delete=False) as f:
 5    print(f'writing to {f.name}')
 6    f.write(b'oneoffcoder.com')
 7
 8# read from temporary file
 9with open(f.name, 'r') as rf:
10    print(rf.read())
11
12# create a temporary directory
13with tempfile.TemporaryDirectory() as d:
14    print(f'created temporary directory {d}')