Making Use of /scratch

Making Use of Local /scratch

If a job produces large files and runs only at one worker node, it is recommended to use a local scratch directory (/scratch) available at a local disk of an assigned worker node.
It is necessary to add a requirement for size and type of a scratch directory to the qsub command. No scratch is set as a default.

#PBS -l select=1:ncpus=8:mem=20gb:scratch_local=50gb

Upon a job start, the batch system creates a unique subdirectory at a local scratch directory,

/scratch*/username/job_JobID.meta-pbs.metacentrum.cz

pointed to by the $SCRATCHDIR variable.

In case you need to check your temporary files at the local scratch subdirectory, log in to the assigned node first. The scratch subdirectory is not accessible directly from the login server where a job has been submitted.

Note: The /scratch*/username directory at worker nodes is NOT WRITABLE - it has permissions 555.

To make use of the scratch directory, modify your job script in following way:

• copy input files to $SCRATCHDIR
• run computations in $SCRATCHDIR
• copy back output files from $SCRATCHDIR to the working directory at /storage/locality/, where the job has been submitted ($PBS_O_WORKDIR)
• clean up $SCRATCHDIR - use a clean_scratch utility to perform safe scratch cleanup. Add the following lines to your job script:

trap ‘clean_scratch’ TERM EXIT ... cp results /storage/... || export CLEAN_SCRATCH=false

Temporary data in a local /scratch are automatically removed in one or two weeks.

Node Type	Total /scratch
luna65-95	800 GB
luna96-99	1.7 TB
luna100-105	3.4TB
luna201-206	800 GB

Making Use of Shared /scratch

The clusters Luna2019, Luna2021 and Luna2022 have no shared scratch. It is possible to make use only of a local scratch directory available at each worker node or of a user directory /storage/praha6-fzu/home/username/ NFS mounted from a local disk array.