How to run Q-Chem on USC HPCC

First, you need to secure an allocation on HPCC. Check if your group already has an allocation and ask your professor to add you to the list of users. If you do not have one --- request HPC account and allocation (this should be done by the PI). If you are enrolled in Chem545, your name will be added to the class HPCC user list. Once you have the allocation, email to hpc@usc.edu and request to be added to the qchem group. Once this is done, you can submit jobs either by using IQmol graphic interface (which you need to install on your laptop or desktop) or by ssh-ing to HPCC and using a PBS script (this requires a basic knowledge of Unix OS).

Using IQmol

  • Download IQmol graphic interface from the IQmol website.
  • Follow the steps in this tutorial to configure IQmol to submit jobs to HPCC.

From HPCC command-line using PBS (deprecated, will be disabled soon)

  • Login to USC HPCC using your USC/HPC account.
  • Create your Q-Chem input file:
    my_input.inp
  • Create PBS run script
    qcpbs.run
    with the following content making necessary adjustments to queue, walltime, etc. The basic script is shown below (thanks Cesar Sul for useful feedback). The user may want to change the number of cores or Q-Chem version.
    #!/bin/bash
    #PBS -l walltime=00:30:00
    # Set the number of cores (in this example it is 4)
    #PBS -l nodes=1:ppn=4
    
    # Change the default version to an older if something does not work in Q-Chem
    source /usr/usc/qchem/default/setup.sh
    
    cd $PBS_O_WORKDIR
    # DO NOT FORGET TO CHANGE THE NUMBER OF CORES HERE!!!!
    # Set the number of cores (-nt the_number_you_need)
    qchem -nt 4 -save test_2_mp4.inp test_2_mp4.inp.out
    # Uncomment if you need scratch files for NO, NDO, or NTO:
    #cp -R "$TMPDIR" "$PBS_O_WORKDIR"
           
    Advanced version (for developers):
    #!/bin/bash
    #PBS -q default
    #PBS -l walltime=00:30:00
    #PBS -l ncpus=1
    
    export QC=/usr/usc/qchem/default
    export QCAUX=$QC/qcaux
    export QCPLATFORM=LINUX_Ix86_64
    export QCRSH=ssh
    export PATH=$QC/bin:$PATH
    export QCSCRATCH=$TMPDIR
    
    cd $PBS_O_WORKDIR
    qchem -nt 1 -save my_input.inp my_output.out
    cp -R "$TMPDIR" "$PBS_O_WORKDIR"
              
  • Submit your PBS job:
    qsub qcpbs.run
  • To run Q-Chem in parallel change
    -nt 1
    to
    -nt N
    in the above script (N is the number of CPU cores; typical values are 4-32). This option should be used only for jobs that take more than 15 min. Consult Q-Chem manual for more information.
  • Note
    -save
    option in the above script: it means that additional files will be copied to your HPCC home, in IQmol's working directory. This option is needed for wave function analysis.

From HPCC command-line using SLURM (new)

  • Login to USC HPCC using your USC/HPC account.
  • Create your Q-Chem input file:
    my_input.inp
  • Create SLURM run script
    qc.run
    with the following content making necessary adjustments to queue, walltime, etc. The user may want to change the number of cores or Q-Chem version. These are tested only in the testing environment, provided by HPCC. Changes may happen in future due to queue structure, different defaults, etc.
    #!/bin/bash
    # Set the number of cores (in this example it is 4)
    #SBATCH --ntasks=4
    #SBATCH --mem=4000MB
    
    source /usr/usc/qchem/default/setup.sh
    export QCSCRATCH=$TMPDIR
    
    cd $SLURM_SUBMIT_DIR
    hostname
    # DO NOT FORGET TO CHANGE THE NUMBER OF CORES HERE!!!!
    # Set the number of cores (-nt the_number_you_need)
    qchem -nt 4 -save my_input.inp my_input.inp.out
    # Uncomment if you need scratch files for NO, NDO, or NTO:
    #cp -R "$TMPDIR" "$SLURM_SUBMIT_DIR"
           
    Advanced version (for developers):
    #!/bin/bash
    #SBATCH --ntasks=4
    #SBATCH --job-name=my_task
    #SBATCH --output=my_task.stdout
    #SBATCH --error=my_task.error
    #SBATCH --mem=4000MB
    
    #Depending on your build, you may change QC and QCAUX
    export QC=/usr/usc/qchem/default
    export QCAUX=$QC/qcaux
    export QCPLATFORM=LINUX_Ix86_64
    export QCRSH=ssh
    export PATH=$QC/bin:$PATH
    export QCSCRATCH=$TMPDIR
    
    cd $SLURM_SUBMIT_DIR
    hostname
    qchem -nt 4 -save my_input.inp my_input.inp.out
    # Uncomment if you need scratch files for NO, NDO, or NTO:
    cp -R "$TMPDIR" "$SLURM_SUBMIT_DIR"
              
  • Submit your SLURM job:
    sbatch qc.run
  • To run Q-Chem in parallel change
    -nt 1
    to
    -nt N
    in the above script (N is the number of CPU cores; typical values are 4-32). This option should be used only for jobs that take more than 15 min. Consult Q-Chem manual for more information.
  • Note
    -save
    option in the above script: it means that additional files will be copied to your HPCC home, in IQmol's working directory. This option is needed for wave function analysis.

Additional Resources