Tutorial (pysimfs)

This section introduces the use of pysimfs for more complex simulation tasks.

Building a pipeline

You have seen the basic structure of a SiMFS-Tk simulation in the Introduction and Quickstart (SiMFS-core) sections. So far the simulations have been a step-by-step procedure with a lot of intermediate file clobber. If you are familiar with the UNIX file system, you might have guessed, that you can avoid the large amount of data on disk by using pipes. Pipes are file-like obejects that allow concurrent writing and reading to pass information from one process to the other.

The manual approach would be to configure two processes like simfs_dif and simfs_exi to write to and read from the same file called coord_pipe. Then you create a named pipe of this name and start the processes in parallel:

$ mkfifo coord_pipe
$ simfs_dif < dif.json > dif.log.json & simfs_exi < exi.json > exi.log.json

The two processes run simultaneously passing coordinates from simfs_dif to simfs_exi continuously without using diskspace for coordinate data. Only the resulting excitaion data file will be created (unless there is more piping).

This approach is pretty powerful in that it saves diskspace, avoids unnecessarily large intermediate data files and allows for better usage of multiple cores. It is however quite cumbersome to manage the named pipes yourself. As soon as one connection within a complex pipeline is not set correctly, the whole run blocks and leaves partial processes and files behind – a huge mess! Therefore we have a little python package that helps you set up and run your SiMFS-components, while managing the required pipes for parallel execution for you! The next section will show you how to use pysimfs to run a similar simulation like the basic commandline example from python.

Warning

Pysimfs is a python package designed to be used in python scripts and jupyter-notebooks This tutorial requires a basic understanding of python, numpy and matplotlib.

Pysimfs startup

Once you have the pysimfs package installed, you can import it. It will try to locate the SiMFS-Tk executables in its basepath. If there is an error, check the location of your SiMFS-core installation and adapte the path in the pysimfs package’s __init__.py.

The import should look like this:

>>> import pysimfs as simfs
    All simfs components found in /opt/SiMFS-Tk/SiMFS-core/build/src/components/.

Configuring components

Each simfs_xxx executable has a pysimfs object to control it. To create a diffusion component, create a pysimfs.Diffusion object like this:

>>> dif = simfs.Diffusion()
>>> dif.params
    {'collision_output': '__collisions__',
     'coordinate_output': '__coordinates__',
     'diffusion_coefficient': 1e-10,
     'experiment_time': 1.0,
     'half_height': 1e-06,
     'increment': 1e-07,
     'radius': 1e-06,
     'seed': 477755232}

So far no simulation ran. You can inspect (or change) the current paramters of the component in its params dictionary. Every component constructor takes keyword argument that map exactly to the toplevel json members of the manages SiMFS-somponent. To create the diffusion example again:

>>> import os
>>> dif = simfs.Diffusion(
...    collision_output=os.devnull,
...    coordinate_output='coords.dat'
...    diffusion_coefficient=4.35e-10
... )
>>> dif.params
    {'collision_output': '/dev/null',
     'coordinate_output': 'coords.dat',
     'diffusion_coefficient': 4.35e-10,
     'experiment_time': 1.0,
     'half_height': 1e-06,
     'increment': 1e-07,
     'radius': 1e-06,
     'seed': 2235270572}

This is simple a convenient wrapper around the process of getting the defaults with the list option, save it to a file, edit it and reapply the changes.

Running a simulation

So far no simulation run. To run the single component, create a Simulation and add the configured component to it:

>>> sim = simfs.Simulation()
>>> sim.add(dif)

There might be some messages about already existing folders, when you run this multiple times. You can ignore this for now. To start the simulation simply call run on the simulation:

>>> log = sim.run()
    started /opt/SiMFS-Tk/SiMFS-core/build/src/components/mol/simfs_dif
    Simulation completed after 1.86 seconds.

You get list of all paramter logs and error messages of all components. The output reports which executables were started and finally how long the simulatio took. The file coords.dat will be created in your current direcory. To quickly load the simulation results into your python session, use the get_results function of the simulation:

>>> res = sim.get_results()

res now contains a dictionary of numpy arrays that contain the result data.

Chaining components

We could now continue to run the next component in a new simulation, reading the new coords.dat file. However pysimfs is capable of handling more than one component per simulation. Take a look at this new simulation:

>>> sim = simfs.Simulation()
>>> sim.add(
        simfs.Diffusion(
            coordinate_output='coords',
            collision_output=os.devnull,
            diffusion_coefficient=4.35e-10
        )
    )
>>> sim.add(
        simfs.Excitation(
             input='coords',
             output='exi.dat'
        )
    )
>>> log = sim.run()
    started /opt/SiMFS-Tk/SiMFS-core/build/src/components/fcs/simfs_exi
    started /opt/SiMFS-Tk/SiMFS-core/build/src/components/mol/simfs_dif
    Simulation completed after 2.93 seconds.
>>> res = sim.get_results()

Two components were started simultaneously. Pysimfs detected that dif and exi share an output/input pair and created the requirede pipe for us. As a result only the exi.dat file is generated. The intermediate coords are gone and don’t occupy any disk space.

Utilities

With pysimfs it is much easier to design and run more complex simulations. Checkout the example notebooks for more complex simulations. There are some more helper functions included in pysimfs that make the simulatio process easier:

Default parameters

If you work in a jupyter notebook context, it can be helpful to have explicit parameter dictionaries in some cases. To quickly load a default parameter set from a component, you can use the simfs_default magic:

%simfs_default dif

As a result, a default paramter dictionary is written to the current cell.

Warning

Note that the magic overwrites the content of the current cell.

You can pass a paramter dictionary to a component by using **kwargs:

dif = simfs.Diffusion(**dif_params)

Context manager

Since pysimfs interacts a lot with the filesystem, there can be some accumulation of old intermediate files like named pipes. Pysimfs uses temporary folders to manage its files, but does not clean them up by default.

An easy way to manage the lifetime of these files is to use the context manager capability of Simulation:

import os
from pysimfs import *
with Simulation() as S:
    S.add(Diffusion(coordinate_output='coords', collision_output=os.devnull)
    S.add(Excitation(input='coords', output='exi.dat')
    log = S.run()
    res = S.get_results()

This idiom wraps the simulation setup and run into a context that handles the cleanup of temporary files when done.

Grid data

The simfs_img and simfs_pre component produce numeric data on a grid in a binary format. Pysimfs includes a loader for these files.

img = simfs.util.GridData('image.dat')

Checkout the notebooks for further usage examples.