Custom Frames Paths

Added in 0.10

What is Custom path.yaml

The custom_path.yaml file should be placed in shared/smartscope and is meant to customize the directories and names of where the raw movie frames will be saved.

File synthax

Changing frames paths for a single detector

1. Find detector id

A specific path can be set for each Detector in the database. First, you'll need to find you detector_id from the admin portal.

1. Find your detector entry in the admin portal and open up the settings:
2. The detector id will show up in the url of the page. In this example: http://127.0.0.1:8000/admin/API/detector/4/change/ The detector id is 4

2. Create an entry for your detector in custom_paths.yaml

See the example below:

detector_id_4:
    ...

The first line shows detector_id_? where he ? is the specific id of the detector we're trying to customize.

Changing default path for all detector

1. Create an detector_default in custom_paths.yaml

See the example below:

detector_default:
    ...

The first line shows detector_default

Order of operations

The file may have both a detector_default and specfic detector_id_? entries. The detector_id_? will take precedence when present and fallback to detector_default

Customizing the path

Then, we specify the frames_directory_structure as a list. - For Gatan detectors, this list will get concatenated into subdirectories - For Falcon detectors, this list will get concatenanted into a single directory with each entry separated with an underscore.

detector_id_4:
  frames_directory_structure:
   - {{session_id.working_directory}}
   - {{position}}_{{name}}

The example above would store the frames in something like: 20260129_test_session/1_box1_1

Using the curly bracket synthax

The example above just lists the default behavior prior to customization. Each item encapsulated between double curly brackets {{item}} will be parsed according to their respective database entry, related to the Grid instance. Many items can be specified. Here's a non-exhaustive list:

Field	Description
name	Name of the grid as shown in the web_ui without the number
position	Position number of the grid in the cassette
grid_id	id of the grid. Somewhat radomizer string of alphanumerial with parts of the name
collection_mode	whether the grid is labelled as a screening or collection
session_id.date	Date of the session in YYYYMMDD format
session_id.session	Name of the session
session_id.working_directory	Shortcut for the last two fields YYYYMMDD_SESSIONNAME
session_id.group	Name of the group associated with the session
session_id.user	Username of the user associated with the session
session_id.user.first_name	First name of the user associated with the session
session_id.user.last_name	Last name of the user associated with the session
session_id.microscope_id.name	Name of the microscope
session_id.detector_id.name	Name of the detector

Examples

1. Storing the frames per groups and collection types:

   detector_id_4:
     frames_directory_structure:
      - {{session_id.group}}
      - {{collection_type}}
      - {{session_id.working_directory}}
      - {{position}}_{{name}}
   

   Will store the frames as Testing/screening/20260129_test_session/1_box1_1

2. Adding the username to the session directory

   detector_id_4:
     frames_directory_structure:
      - {{session_id.date}}_{{session_id.user}}_{{session_id.session}}
      - {{position}}_{{name}}
   

   Will store the frames as 20260129_jdoe_test_session/1_box1_1

Validating your file

Note

For this commad to work, you will need at least one existing grid for that detector in the database. It does not have to have run yet but it needs to exist.

After setting up the file, you may run a test command to get an idea of where the files would be saved from the SmartScope and SerialEM point of view for a given grid that used this detector.

./smartscope.sh run validate_custom_paths DETECTOR_ID

Output should look like this with default configuration:

2026-01-29 11:53:09 [Smartscope.core.test_commands:265 -     INFO]   Validating custom paths for detector 3 with grid 1_grid1
frames_directory_structure=['{{session_id.group}}', '{{session_id.working_directory}}', '{{position}}_{{name}}']
2026-01-29 11:53:09 [Smartscope.core.test_commands:267 -     INFO]   Generated path for grid 1_grid1: testing/20250313_test/1_grid1
2026-01-29 11:53:09 [Smartscope.core.test_commands:269 -     INFO]   Full path from smartscope POV: /mnt/fake_scope/movies/testing/20250313_test/1_grid1
2026-01-29 11:53:09 [Smartscope.core.test_commands:271 -     INFO]   Full path from SerialEM POV: X:\testing\20250313_test\1_grid1

Output should look something like this if LOGLEVEL=DEBUG in smartscope.conf:

2026-01-29 11:53:09 [Smartscope.core.main_commands:42 -    DEBUG]   Running command=validate_custom_paths 3
2026-01-29 11:53:09 [Smartscope.core.test_commands:265 -     INFO]   Validating custom paths for detector 3 with grid 1_grid1
frames_directory_structure=['{{session_id.group}}', '{{session_id.working_directory}}', '{{position}}_{{name}}']
2026-01-29 11:53:09 [Smartscope.core.frames:45 -    DEBUG]   Parsed {{session_id.group}} to testing
2026-01-29 11:53:09 [Smartscope.core.frames:45 -    DEBUG]   Parsed {{session_id.working_directory}} to 20250313_test
2026-01-29 11:53:09 [Smartscope.core.frames:45 -    DEBUG]   Parsed {{position}} to 1
2026-01-29 11:53:09 [Smartscope.core.frames:45 -    DEBUG]   Parsed {{name}} to grid1
2026-01-29 11:53:09 [Smartscope.core.test_commands:267 -     INFO]   Generated path for grid 1_grid1: testing/20250313_test/1_grid1
2026-01-29 11:53:09 [Smartscope.core.test_commands:269 -     INFO]   Full path from smartscope POV: /mnt/fake_scope/movies/testing/20250313_test/1_grid1
2026-01-29 11:53:09 [Smartscope.core.test_commands:271 -     INFO]   Full path from SerialEM POV: X:\testing\20250313_test\1_grid1