Filesystem directories¶
This page describes the filesystem directories available in the Notebook Aspect.
| Directory | Private | Persistent | Writeable |
|---|---|---|---|
| $HOME | ✅ | ✅ | ✅ |
| /datasets | ✅ | ||
| /project | ✅ | ✅ | |
| /scratch | ✅ |
$HOME¶
Your home directory is just that.
The $HOME directory is also the base directory for the JupyterLab file browser.
Notebooks and other files need to be inside the $HOME directory (or a subdirectory) for you to see and open them from the file browser.
Currently there is a 100 GB quota for $HOME.
There is also a limit of 102400 files and subdirectories (inodes) within $HOME.
You can use /project for larger datasets.
Note
If you also have an account on lsst-dev,
you can access your Notebook Aspect home directory via $HOME/jhome/ from lsst-dev.
Preinstalled subdirectories¶
Within the $HOME directory a number of directories are currently pre-created for new users.
The ~/notebooks directory¶
The ~/notebooks directory is intended as the top of a subtree within which you can organize your personal notebooks.
Within this directory, two items are pre-created:
- A clone of the lsst-sqre/notebook-demo repository is created from GitHub at
~/notebooks/notebook-demo. Theprodbranch of this repository is regularly updated from GitHub in this clone. See the Demo notebooks page to learn more. - You can use the ~/notebooks/.user_setups file to configure the environment your notebooks run in.
The ~/DATA directory¶
The ~/DATA directory is pre-created, empty, as a place for tutorial notebooks to load input datasets.
If you delete it, some notebooks will attempt to re-create it, while others may fail.
The ~/WORK directory¶
The ~/WORK directory is pre-created, empty, as a place for some tutorial notebooks to write outputs (e.g., “rerun” directories).
If you delete it, these notebooks may fail.
The ~/dask directory¶
The ~/dask directory is pre-created and holds an automatically updated Dask configuration file, ~/dask/dask_worker.yml.
This file is recreated on each login, populated with a template that builds dask nodes of the appropriate size and with the appropriate disk mounts.
The ~/idleculler directory¶
The ~/idleculler directory is used for logfile output from the process that watches for idle sessions.
This file, culler.output, may be deleted when it begins to take up a lot of space; it will be automatically recreated.
/datasets¶
This is a read-only directory containing curated Butler data repositories. Each dataset has a README file describing its contents and intended applications. See Common Dataset Organization and Policy in the DM Developer Guide for more information about these datasets.
/project¶
This directory is for shared, uncurated, persistent data. There is no disaster recovery, nor is there an enforced quota or purge policy.
/scratch¶
This directory is for completely transient data. There is no disaster recovery or quota, but there is a purge cycle.