Dictionaries, Files, and the Mode That Deletes Your Data

One character, one disaster#

You have a file of notes you have been adding to all semester. You want to add one more line:

f = open("notes.txt", "w")
f.write("one more line\n")
f.close()

What is in notes.txt now? Decide before you read on.

The reasonable guess is: everything that was there, plus the new line. The code only ever writes one line, so how could it do anything else?

The file now contains one line. A whole semester of notes is gone, and there is no undo.

"w" does not mean “write” in the way we read it. It means truncate this file to nothing, then write. The wipe happens the instant open runs, before a single character of ours is written. The mode we wanted was "a", for append.

Lookup by name instead of by number#

Think of a filing cabinet with a label on the front of every folder. You do not ask for “the fourth folder”. You ask for the folder marked Water, and you get it straight away, whether it is at the front or the back, and whether there are ten folders or ten thousand.

A list is the opposite. A list is a numbered row of lockers, and the only way to ask for something is by its locker number.

A list finds things by position. A dictionary finds them by a key you choose — the key being the label you wrote on the folder:

item = {"name": "Water", "amount": 10}
print(item["name"])
item["amount"] = 20
print(item)

Water
{'name': 'Water', 'amount': 20}

Same square brackets as a list, but what goes inside is a key, not an index — item["name"], never item[0]. Assigning to a key that does not exist adds it; assigning to one that does replaces its value. There is no separate "add" operation.

Keys are usually strings, and they must be immutable — which rules out using a list as a key:

d = {}
d[["a"]] = 1

Traceback (most recent call last):
  File "d.py", line 2, in <module>
    d[["a"]] = 1
    ~^^^^^^^
TypeError: unhashable type: 'list'

"Unhashable" is Python's word for "this could change underneath me, so I cannot use it to file things by." A tuple works; a list does not.

KeyError, and the two ways to avoid it#

item = {"name": "Water"}
print(item["price"])

Traceback (most recent call last):
  File "k.py", line 2, in <module>
    print(item["price"])
          ~~~~^^^^^^^^^
KeyError: 'price'

The key is missing. In real code the cause is almost always a typo, a case mismatch — "Name" is a different key from "name" — or one item in a collection built with a different shape from the rest.

item = {"name": "Water"}
print(item.get("price"))
print(item.get("price", 0))
print("name" in item)

None
0
True

Use the brackets when a missing key means your program is wrong and you want to hear about it. Use .get() when a key is legitimately optional. in asks about keys, never values.

Looping over a dictionary#

d = {"one": 1, "two": 2}
print(list(d.keys()))
print(list(d.values()))
for k, v in d.items():
    print(k, v)

['one', 'two']
[1, 2]
one 1
two 2

.items() hands you both at once, which saves a lookup inside the loop. Since Python 3.7 a dictionary keeps its insertion order — but if you need a particular order, ask for it:

roster = {"Jill": "A", "Jack": "B"}
for k in sorted(roster.keys()):
    print(k, roster[k])

Jack B
Jill A

d = {"a": 1, "b": 2}
del d["a"]
print(d, len(d))

{'b': 2} 1

The counting idiom#

This is the single most reused dictionary pattern, and it is worth memorizing as one line:

words = ["a", "b", "a", "c", "a"]
counts = {}
for w in words:
    counts[w] = counts.get(w, 0) + 1
print(counts)

{'a': 3, 'b': 1, 'c': 1}

Read it right to left: get the current count or zero if this is the first time, add one, store it back. The 0 default is what makes the first sighting of an item work without a special case — write it with plain brackets and the first counts[w] + 1 raises KeyError.

A list of dictionaries#

Records with named fields, in order — the shape most small data sets take:

supplies = [{"name": "Water", "amount": 1}, {"name": "Radio", "amount": 2}]
for it in supplies:
    print(it["name"], it["amount"])

Water 1
Radio 2

The loop hands you one dictionary per pass; the keys read its fields. Every record needs the same keys — one item missing "amount" stops the loop with a KeyError partway through, after some output has already been printed.

Reading and writing files#

An open file is like a running tap. Leave it running and you get problems that show up somewhere else entirely, long after the line that caused them. Something has to turn it off.

with is the thing that turns it off. Read with open("demo.txt", "w") as f: out loud as: “open demo.txt for writing, call it f, and close it again when this indented block ends”. The closing is automatic, and it still happens if the code inside stops with an error partway through. You will see with called a context manager, which is just the formal name for something that tidies up after itself like this.

with open("demo.txt", "w") as f:
    f.write("first\nsecond\n")

with open("demo.txt") as f:
    print(f.read())

first
second

Note that .write() does not add a newline — if you want lines, you write the \n yourself.

with open("demo.txt", "w") as f:
    f.write("first\n")

with open("demo.txt", "w") as f:
    f.write("second\n")

with open("demo.txt") as f:
    print(f.read())

second

Same code with "a" on the second open keeps both:

with open("demo.txt", "w") as f:
    f.write("first\n")

with open("demo.txt", "a") as f:
    f.write("second\n")

with open("demo.txt") as f:
    print(f.read())

first
second

Reading line by line, and the extra blank lines#

Looping over an open file gives you one line at a time — and each one arrives with its newline still attached:

with open("demo.txt", "w") as f:
    f.write("first\nsecond\n")

with open("demo.txt") as f:
    for line in f:
        print(repr(line))

'first\n'
'second\n'

repr() is the trick that shows it. Because the line already ends in a newline and print adds another, output comes out double-spaced. .strip() fixes it:

with open("demo.txt", "w") as f:
    f.write("first\nsecond\n")

with open("demo.txt") as f:
    for line in f:
        print(line.strip())

first
second

.readlines() gives the whole file as a list in one go — convenient for a small file, and the newlines are still there:

with open("demo.txt", "w") as f:
    f.write("first\nsecond\n")

with open("demo.txt") as f:
    print(f.readlines())

['first\n', 'second\n']

Combine that with .split() and you can read a data file into fields: loop the lines, strip each one, split it on the separator, convert what needs converting.

Exceptions#

An exception is Python stopping the program because it has hit something it cannot do. The block of text it prints on the way out is the traceback. Neither word means anything more complicated than that.

A missing file stops the program:

f = open("nope.txt")

Traceback (most recent call last):
  File "o.py", line 1, in <module>
    f = open("nope.txt")
        ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'nope.txt'

try/except lets you carry on instead:

try:
    f = open("nope.txt")
except FileNotFoundError:
    print("Could not open the file.")

Could not open the file.

Name the exception you expect. A bare except: swallows everything — including the typo in the line below it, which then produces no error and no output, and leaves you debugging a program that quietly does nothing.

Catching the exception object gives you the message:

try:
    n = int("hi")
except ValueError as e:
    print("ValueError:", e)

ValueError: invalid literal for int() with base 10: 'hi'

That is the exception behind every int(input()) on a user who typed a word, which makes it the one worth handling in a first course.

The full form has four parts:

def divide(x, y):
    try:
        result = x // y
    except ZeroDivisionError:
        print("cannot divide by zero")
    else:
        print("answer is", result)
    finally:
        print("done")

divide(10, 2)
divide(10, 0)

answer is 5
done
cannot divide by zero
done

else runs only when nothing was raised. finally runs either way — note done appears under both calls. Keep the try block down to the line that can actually fail and put the rest in else; a large try hides which line you were guarding against.

Where these go wrong#

• Indexing a dictionary by number. item[0] looks for a key 0, not the first entry.
• Case mismatch in a key. "Name" and "name" are different keys. A very common KeyError.
• counts[w] + 1 on the first sighting. KeyError. Use counts.get(w, 0) + 1.
• Opening for writing to inspect a file. "w" empties it on open. Use "r", or "a".
• Forgetting .strip() on file lines. Double-spaced output, and comparisons that fail because of an invisible \n.
• Expecting .write() to add newlines. It does not. Write \n yourself.
• A bare except:. Hides real bugs, including typos on the very next line.
• A try block wrapping half the program. You lose track of which line you were guarding.
• Assuming every record has the same keys. One odd item raises partway through, after output has already been printed.

Frequently asked questions#

Something here not clear? A topic you wish we covered? Tell us. We read every message, and a request is the fastest way to get a guide written — several of these exist because somebody asked.