ENG-CANSAT-PICO-LOG

MicroPython File System

MicroPython board does fits an internal file system. This means that Python language on a MicroPython board can also manage files and directories! Exactly like python does on a computer :-).

On a MicroPython board, you can manage files and directories. Creating them, deleting them, adding data to them, etc.

As any good implementation of Python, MicroPython can manage text files and binary files.

Data Logging with MicroPython

Doing data logging with MicroPython is quite easy. You just open a file and write data into it like you will do with any Python code.

Writing data to text file

The following example adds lines to the 'file.txt'. The with statement will automatically close the file when the code gets out of the with scope.

with open( 'file.txt', 'a' ) as _file:
    _file.write( "Hey, it is MicroPython\r\n" )

Notes:

• The 'a' parameter in the open() call stand for "append". So executing several times the code will append the same text again and again to the file.
• IF the file doesn't exists yet THEN it will be automatically created.
• Use the 'w' parameter (instead of 'a') to overwrite the file content.
• As the write() just writes bytes then the the carriage return and line feed (\r\n) must also be written.

The following screen capture do show the 'file.txt' on the pico file system (do not hesitate to refresh the file list) and the file content is displayed on the file pane.

Reading data from text file

The following script read the content of the "file2.txt" text file and displays it on the REPL output.

with open( 'file2.txt', 'r' ) as _file:
    s = _file.readline()
    while s:
        print( s.strip('\r\n') ) 
        s = _file.readline()

Notes:

• the file mode 'r' used in the open() reads text file.
• the readline() read and return a line from the file (including the carriage return & line feed.
• the strip('\r\n') remove the carriage return & line feed from the readed line. Otherwise it will also be printed to the REPL (which result into a double line feed).

File-systems, Writing and data lost

The computer file-system:

When working with file on a computer it is essential (even critial) to close the file to be ensured that all data are written to the file-system.

Ensuring good performance on a computer do involves lot buffering mechanism at software and hardware levels.

Cutting the power on a computer usually results into missing data or corrupted files.

Note: this is not true for databases which ensure and 200% the log file writing before applying changes to the database.

SD Card file system:

Regardless of the platform using the SD card, the content and stability of a SD card is veryyyyyyy sensitive to the open/close operation on the file-system. Users of the Raspberry-Pi nano computer knows that powering of a Pi during SD write operation could completely corrupt the file-system. In some rare situation the SD card can be made completely unreadable.

Using SD-Card with microcontroler is not recommended when writing operation are involved. Using SD in write mode implies electronic skills to spare some "emergency power" for closing files on the SD card before the final shutdown.

MicroPython file-system:

The since MicroPython 1.12 (2019), the file-system is based on LittleFS.

LittleFS is a flash-based device filesystem. It has been designed to be more resistant to filesystem corruption and power failures resilient. Unless other file-systems, the RAM used by LittleFS does not depend on the file-system size (it is said "RAM bounded" without RAM allocation overflow). See LittleFS Github for more information.

So, even if LittleFS do have minimal buffering, it is possible to force writing to the flash ( with flush() ) to ensure minimal data lost in case of issue. So the user code can rely on better file-system stability and better control over forced writing operation.

The following example just flush() writes at every line written to the file.

Power can be lost at every moment, the flushed writes are safely stored.

import time
with open( 'file2.txt', 'a' ) as _file:
    for i in range( 50 ):
        print( i )
        _file.write( 'Hey, it is MicroPython %i\r\n' % i )
        _file.flush() # Force writing operation
        time.sleep( 1 )

Please note: abusing the flush() operation will slow down the code and will degrade flash faster (Flash Memory cells do have a maximum write cycles).

Ressources

• Basic "operating system" services (micropython.org)
  Usual OS functions + file-system mounting, formating, block devices, etc
• Working with filesystems (micropython.org)

---

Written by Meurisse D. for MCHobby