Hi all,
I have had trouble linking CESM with an external library (the RTTOV radiative transfer model) when building on Derecho (I didn't have this issue on Cheyenne but am not sure how relevant that is). I have done my best to describe my issue and code changes in a useful way below, including code and details where most relevant. I'm hoping to eventually pull this new code into CESM/CAM and would love some help as I try to handle a few final bugs that I haven't been able to figure out on my own.
Problem: CESM build fails when RTTOV is built against the HDF5 library and/or RTTOV is built using openmp (error messages during the model build are attached).
A note: When RTTOV is not built with hdf5 or openmp I can link it to CESM on Derecho without errors, but unfortunately I need the hdf5 functionality (openmp is not as important).
My intuition: There are additional flags needed when compiling CESM against RTTOV with the openmp and hdf5 packages. I think that maybe I am trying but not correctly adding those flags ("-lz" and "-qopenmp") to the CESM build.
CESM source code here: /glade/u/home/jonahshaw/w/CESM_source/cesm2_3_alpha17b_rttovhdf (version_info.txt is attached)
Example case here: /glade/u/home/jonahshaw/Cases/CESM2/cesm2_3_alpha17b_rttovhdf/20240327_163959.FHIST.f19_f19_mg17.rttov_test
My code changes (happy to provide these in more detail if helpful):
Changes to the CESM Makefile (cime/CIME/Tools/Makefile):
# Additionally include RTTOV if asked via CAM_CONFIG_OPTS.
ifeq ($(findstring -rttov,$(CAM_CONFIG_OPTS)),-rttov)
ULIBDEP += $(COSP_LIBDIR)/librttov_wrapper.a $(COSP_LIBDIR)/librttov_mw_scatt.a $(COSP_LIBDIR)/librttov_brdf_atlas.a $(COSP_LIBDIR)/librttov_emis_atlas.a $(COSP_LIBDIR)/librttov_other.a $(COSP_LIBDIR)/librttov_parallel.a $(COSP_LIBDIR)/librttov_coef_io.a $(COSP_LIBDIR)/librttov_hdf.a $(COSP_LIBDIR)/librttov_main.a
endif
Changes to files in components/cam/bld:
# Turn on RTTOV if passed the keyword. The cosp-rttov Makefile copies rttov libraries to the $cosp_libdir path
if ($rttov) {
$ldflags .= " -L$cosp_libdir -lrttov13_wrapper -lrttov13_mw_scatt -lrttov13_brdf_atlas -lrttov13_emis_atlas -lrttov13_other -lrttov13_parallel -lrttov13_coef_io -lrttov13_hdf -lrttov13_main -lhdf5_hl_fortran -lhdf5_hl -lhdf5_fortran -lhdf5 ";
# Additional flags needed when RTTOV is compiled with HDF5 (-lz) and openmp (-qopenmp).
$ldflags .= " -lz -qopenmp";
$cfg_ref->set('ldflags', $ldflags);
print "Adding rttov libraries as dependencies in ldflags.\n";
}
Changes to cam source code:
Changes to COSP source code:
For this issue, I think that the relevant changes are in the CESM Makefile and the cam configure file.
I have a few other questions related to building CESM with this new code but wanted to focus on this single issue here. It would be great to talk through this and other questions with a software engineer and I would be happy to set aside time for a call or come up to the Mesa Lab.
Thanks!
Jonah
I have had trouble linking CESM with an external library (the RTTOV radiative transfer model) when building on Derecho (I didn't have this issue on Cheyenne but am not sure how relevant that is). I have done my best to describe my issue and code changes in a useful way below, including code and details where most relevant. I'm hoping to eventually pull this new code into CESM/CAM and would love some help as I try to handle a few final bugs that I haven't been able to figure out on my own.
Problem: CESM build fails when RTTOV is built against the HDF5 library and/or RTTOV is built using openmp (error messages during the model build are attached).
A note: When RTTOV is not built with hdf5 or openmp I can link it to CESM on Derecho without errors, but unfortunately I need the hdf5 functionality (openmp is not as important).
My intuition: There are additional flags needed when compiling CESM against RTTOV with the openmp and hdf5 packages. I think that maybe I am trying but not correctly adding those flags ("-lz" and "-qopenmp") to the CESM build.
CESM source code here: /glade/u/home/jonahshaw/w/CESM_source/cesm2_3_alpha17b_rttovhdf (version_info.txt is attached)
Example case here: /glade/u/home/jonahshaw/Cases/CESM2/cesm2_3_alpha17b_rttovhdf/20240327_163959.FHIST.f19_f19_mg17.rttov_test
My code changes (happy to provide these in more detail if helpful):
Changes to the CESM Makefile (cime/CIME/Tools/Makefile):
- Add dependencies on several RTTOV libraries.
# Additionally include RTTOV if asked via CAM_CONFIG_OPTS.
ifeq ($(findstring -rttov,$(CAM_CONFIG_OPTS)),-rttov)
ULIBDEP += $(COSP_LIBDIR)/librttov_wrapper.a $(COSP_LIBDIR)/librttov_mw_scatt.a $(COSP_LIBDIR)/librttov_brdf_atlas.a $(COSP_LIBDIR)/librttov_emis_atlas.a $(COSP_LIBDIR)/librttov_other.a $(COSP_LIBDIR)/librttov_parallel.a $(COSP_LIBDIR)/librttov_coef_io.a $(COSP_LIBDIR)/librttov_hdf.a $(COSP_LIBDIR)/librttov_main.a
endif
Changes to files in components/cam/bld:
- components/cam/bld/configure
- Update ldflags to include the RTTOV libraries. The RTTOV documentation says this is how to link the code and the normal cosp library is linked in this way. But the HDF5 libraries (-lhdf5_hl -lhdf5_fortran -lhdf5) do not sit in $cosp_libdir, so I am not sure I have done this correctly.
# Turn on RTTOV if passed the keyword. The cosp-rttov Makefile copies rttov libraries to the $cosp_libdir path
if ($rttov) {
$ldflags .= " -L$cosp_libdir -lrttov13_wrapper -lrttov13_mw_scatt -lrttov13_brdf_atlas -lrttov13_emis_atlas -lrttov13_other -lrttov13_parallel -lrttov13_coef_io -lrttov13_hdf -lrttov13_main -lhdf5_hl_fortran -lhdf5_hl -lhdf5_fortran -lhdf5 ";
# Additional flags needed when RTTOV is compiled with HDF5 (-lz) and openmp (-qopenmp).
$ldflags .= " -lz -qopenmp";
$cfg_ref->set('ldflags', $ldflags);
print "Adding rttov libraries as dependencies in ldflags.\n";
}
- components/cam/bld/namelist_files/namelist_definitions.xml
- Add namelist variables used in the new code.
- components/cam/bld/config_files/definition.xml
- Add options so that RTTOV can be built like cosp using CAM_CONFIG_OPTS
Changes to cam source code:
- components/cam/src/physics/cam/cospsimulator_intr.F90
- Since my error is in the build step I do not think that these changes are relevant to this problem.
Changes to COSP source code:
- See git repository here: https://github.com/jshaw35/COSPv2.0/tree/cesm2.2.0_rel_cosp_rttov
For this issue, I think that the relevant changes are in the CESM Makefile and the cam configure file.
I have a few other questions related to building CESM with this new code but wanted to focus on this single issue here. It would be great to talk through this and other questions with a software engineer and I would be happy to set aside time for a call or come up to the Mesa Lab.
Thanks!
Jonah