iOpenShell » Q-Chem specific questions » MPI and OpenMP not using multiple nodes

MPI and OpenMP not using multiple nodes

Page: 1

Author Post
Member
Registered: Aug 2017
Posts: 1
Hi All,

I am trying to use hybrid MPI+OpenMP parallelization for DFT on Q-CHEM 4.4. My problem is that when I try to use multiple nodes, I find that QCHEM only uses one of the nodes. We use the pbs scheduler, so in my job submission script I write:
#PBS -l select=2:ncpus=24:mpiprocs=1
which should request 2 nodes, each with 1 mpiproc and 24 cpus.

I use the following line to run QCHEM:
qchem -pbs -np 2 -nt 24 input OUT
The -pbs flag sets MACHINEFILE=${PBS_NODEFILE} and if I understand the manual correctly this should request 2 MPI processes, each with 24 threads.

Am I missing a flag? What am I doing wrong?

Thanks,
Dahvyd
Member
Registered: Apr 2018
Posts: 1
Location: Bangkok, Thailand
This is very late reply, but it might be useful for other viewer.

To run Q-Chem in hybrid parallel (MPI+OpenMP), I would rather deal with the qchem executable ($QC/exe/qcprog.exe) directly than $QC/bin/qchem script.
The necessary commands that you should put into your PBS shell script are like

#PBS -l select=2:ncpus=24:mpiprocs=1

export QC='where your qchem directory is'
export QCAUX='where your supplemenary file of q-chem is'
export QCSCRATCH='where you want to store the scratch file'
export OMP_NUM_THREADS='number of threads per MPI process'
export QCTHREADS=$OMP_NUM_THREADS

mpirun -machinefile $PBS_NODEFILE -np N $QC/exe/qcprog.exe input.file $QCSCRATCH > output.file

# where N is number of compute node which listed in $PBS_NODEFILE



More details visit this webpage.

Rangsiman

Page: 1

iOpenShell » Q-Chem specific questions » MPI and OpenMP not using multiple nodes