Lesson 2 - Introduction to Working With Files in Python

In the last lesson, Exceptions in Python, we explained exceptions which we need for working with files, because they allow us to respond to possible errors.

Today we'll talk about files in general and write permissions on Windows systems. After reading this Python tutorial, you'll be ready to work with files.

Data, or more precisely objects stored in memory, will, of course, be lost when the application is terminated. If we want to make sure that the data is persistent, we must save it when exiting the program and reload it when loading the program. There are many ways to store application data, each with some advantages and disadvantages. In general, we can store the data of our application as follows:

• Text files with a flat structure (flat files, e.g. .txt and .csv)
• text files with internal hierarchy (e.g. .xml or .json)
• Binary type files (simply move the memory to file)
• Databases

Each way of storing data has its advantages and disadvantages. We'll describe the individual methods in detail in the Python course and try them out. We'll focus on how to save objects to files and then reload them. A separate section is then devoted to databases.

Write Rights on Windows Systems

Since Windows Vista, there's been UAC (User Account Control) in Windows systems. The technology prohibits writing to the system partition of the disk (usually C:\) if writing isn't allowed by the administrator or if the writing isn't performed in user's personal folders. This measure, of course, filled the discussion forums on programming and disrupted a number of applications. However, the important thing is that this measure is correct and only prevents poorly written applications from changing what they like. Linux systems have had similar measures for a long time.

The Windows concept includes user accounts. Each user account is implemented by a folder (usually C:\Users\username). This folder contains, among other things, documents, the desktop and also the application data folder. This folder can be accessed by entering the %appdata% key in the path. Try writing %appdata% in the explorer path to open the folder. You'll see that many applications that you have installed have subfolders in the folder. This is the place for storing your application's settings and other data. Of course, your application will create a folder with its name here and save everything in it. We'll not be limited by UAC and at the same time our application will allow different settings for different users without programming it in Python. %appdata% always points to the appdata of the currently logged in user.

A big problem was that programs stored their data in the program folder. The program worked on the desktop or on old Windows, however, when it was then installed in the Program Files\ folder on disk C, it crashed with an error when writing to disk. Windows has partially emulated this writing for certain applications, but we'll not deal with it. Of course, you can work around this by running the application as an administrator, but that's a bad solution.

In this lesson, we'll show you how to create an appdata folder for our application. In the next lessons, this logic will no longer be included in the code and it'll serve as the simplest possible example of working with files. If you run the application on your computer, you don't have to worry about anything. However, once you publish it, it's a good idea to edit it to use appdata.

So we already know where we'll write. We also mentioned exceptions. So let's move on to working with the files themselves.

IO Stream and open()

In Python there's a stream, which is a general "data stream". We connect to the stream and then can read data from it or write data to it. The stream is theoretically universal, and the same stream can be in RAM or in a file on disk. Working with files is therefore realized just through the stream in the library io. It wouldn't be Python if it didn't make our job any easier, so there's the open() function that opens a file stream for us. Therefore we won't even need the io library.

In this course, we'll deal with one type of file in each lesson and always show you how to work with it. Now let's show the very basis of how to create a folder for our application in the current user's appdata. We'll use the os.mkdir() function to create a folder for us. Next we'll look at the open() function itself. Thus, we can create the data (e.g. settings.dat) using default values or load it. We'll use os.path to work with the path. All three functions/libraries will be explained in detail in a separate lesson, now we'll only show what's necessary. So let's write an application that'll have its own folder in AppData and see if its database.dat file exists:

import os
path = ""
try:
    path = os.path.join(os.getenv("APPDATA"), "ICTdemy")
    if not os.path.exists(path):
        os.mkdir(path)
except:
    print("Failed to create folder {0}, please check your permissions.".format(path))

if os.path.exists(os.path.join(path, "database.dat")):
    try:
        # Place the code to read your settings from the file here
        pass
    except Exception as e:
        print("Error while loading settings: {0}".format(e))
else:
    try:
        # Place the code to create your settings here
        pass
    except Exception as e:
        print("Error while creating settings: {0}".format(e))

First, we save the path to the folder with our application in the text string path. We get the location of appdata using:

os.getenv("APPDATA")

Python will find its way to %appdata% in the environment that'll be something like this (depending on the version of Windows):

C:\Users\your_name\AppData\Roaming

We'll add a folder with the name of our application to the path. I chose the name ICTdemy/, you can of course edit it later. In order to not have to think about whether the paths have a backslash at the end or a forward slash, we'll join the path using the os.path.join() function in the os library (used only if we want to work with files on multiple platforms for example, once we pass "C:\\" as a parameter/argument, the function loses its meaning). So the path now looks like this:

C:\\Users\\your_name\\AppData\\Roaming\\ICTdemy

If the given folder doesn't exist (this is the first launch of the application), we'll create it. Next, let's see if there's a file from which we want to retrieve data. If so, we'll load the data. If not, we'll create the file and write the default data to it. Or we won't even have to create the file, depending on the type of our application.

After running the code, a new folder will appear in %appdata%

So now we have a quick introduction on how to store application data correctly. We also talked about exceptions. I would rather repeat that in the following lessons, for the sake of simplicity, we won't deal with write rights and the programs will only work in the user's folders (e.g. when running from your IDE). So always come back here and edit the code if you want to distribute the finished application.

Did you have a problem with anything? Download the sample application below and compare it with your project, you will find the error easily.