####WRF-CHEM for Coimabtore: An option with AWS and starcluster####

1. Considering the huge time requirement for running wrf chem over Coimbatore with high resolution (It roughly took 8 hour 15 minutes X 7 for domain 4 alone, 6 second time steps with average 50 seconds), Amazon Web Service (AWS) t2.medium instace and StarCluster compute cluster would be a viable option to make wrf-chem run in ‘real time’. ####AWS charges for a typical cluster####

####upgrading aws free tier into ubunut 14.01#### 1. Since access to aws free tier is through SSH, doing
, up-gradation of current Ubuntu release into 14.04 by sudo do-release-upgrade warns about likely failure in ssh during process. It provide fall over by running ssh in port 1022 and asks to open port from firewall by iptables -I INPUT -p tcp --dport 1022 -j ACCEPT

####compile wrf chem on AWS t1.micro instance free tier#### 1. To remove sudo permission problem followed this and command sudo adduser <username> sudo 1. Compiling WRF in a separate EBS of aws gives advantages of attaching it with EC2 cluster whenever necessary. The star cluster program behind the EC2 cluster can be dictated to do so, providing necessary re-usability of the compiled WRF CHEM program in a temporary cluster environment. For a trail, 8 GB magnetic EBS was created and attached to the running free tier instance. AWS noted about the potential renaming of the EBS as follows Note: Newer Linux kernels may rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered here (and shown in the details) is /dev/sdf through /dev/sdp. 1. Based on this page, mounted the created EBS with running free tier instance. The command given was

            lsblk
            sudo file -s /dev/xvdf
            sudo mkfs -t ext4 /dev/xvdf
    # the above command poteinatal wipe out data if it is there in the EBS
            sudo mkdir /home/ubunut/wrf
    sudo mount /dev/xvdf /home/ubunut/wrf

###Compiler setup### 1. The compilation is carried out as GSI compatible WRF CHEM, so maintained all the version and methods for GSI compilations. Made required update-alternatives for using versions of gcc-4.4, g++-4.4 and gfortran-4.7. 1. So these version are installed first by sudo apt-get install gcc-4.4 sudo apt-get install g++-4.4 sudo apt-get install gfortran-4.7 1. To invoke update-alternative for these compilers used below commands, for gcc and g++ sudo update-alternatives –install /usr/bin/gcc gcc /usr/bin/gcc-4.4 60 –slave /usr/bin/g++ g++ /usr/bin/g++-4.4 sudo update-alternatives –install /usr/bin/gcc gcc /usr/bin/gcc-4.8 40 –slave /usr/bin/g++ g++ /usr/bin/g++-4.8 sudo update-alternatives –config gcc

for gfortran

                which gfortran
      # to knwo the link of current used gfortran
                ls -lh /usr/bin/gfortran
                ls -lh /usr/bin/gfortran*
      #to knwo the list of links for each version
                sudo update-alternatives --install /usr/bin/gfortran gfortran    /usr/bin/gfortran-4.8 60
                sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-4.7 40
                sudo update-alternatives --config gfortran

###Environment setup and Compiling NETCDF 3.6.3, openMPI### 1. The environment was set up with this following commands

*environment setup* 

            export DIR=/home/ubuntu/wrf
            export CC=gcc
            export CXX=g++
            export FC=gfortran
            export FCFLAGS=-m64
            export F77=gfortran
            export FFLAGS=-m64

1. The source code of NETCDF 3.6.3 was downloaded by wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-3.6.3.tar.gz, but ends in permission denied. Found that the mounted EBS is in root permission, so has to make it for user by giving command sudo chmod 777 /home/ubuntu/wrf, then the compilation is done by commands

compiling netcdf tar xzvf netcdf-3.6.3.tar.gz cd netcdf-3.6.3 ./configure –prefix=$DIR/netcdf make make install make check export PATH=$DIR/netcdf/bin:$PATH export NETCDF=$DIR/netcdf it ended without any error.

Compiling openMPI Based on this

            wget https://www.open-mpi.org/software/ompi/v1.8/downloads/openmpi-1.8.1.tar.gz
            tar -xvf openmpi-1.8.1.tar.gz
            cd openmpi-1.8.1.tar.gz    
            ./configure --prefix="/home/$USER/.openmpi"
            make
            sudo make
        export PATH="$PATH:/home/$USER/.openmpi/bin"
            export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/$USER/.openmpi/lib/"

it ended without any error.

###Compiling WRF CHEM 3.4.1### 1. The source code of WRF 3.4.1 and WRF CHEM 3.4.1 was downloaded using wget and unzippign using gunzip WRFV3.4.1.TAR.gz and tar -xf WRFV3.4.1.TAR and copied the chem folder inside WRFV3. 1. Environment setup for wrf chem was done based on this and rechecked with echo, such as echo $WRF_CHEM etc

            export NETCDF=$DIR/netcdf
            export JASPERLIB=/usr/lib
            export JASPERINC=/usr/include
            export WRF_EM_CORE=1
            export WRF_NMM_CORE=0
            export WRF_CHEM=1
            export WRF_KPP=0
            export WRFIO_NCD_LARGE_FILE_SUPPORT=1

Then given command of ./configure and edited the file configure.wrf for the line 119 to 120 into

          FORMAT_FIXED    =       -ffixed-form -cpp
    FORMAT_FEE     =       -ffree-form -ffree-line-length-none -cpp 

from

        FORMAT_FIXED    =       -ffixed-form 
          FORMAT_FEE     =       -ffree-form -ffree-line-length-none

1. then given the command of ./compile em_real >& log.compileA1, it ran for 1 hour and 30 minutes, created all the exe files, subsequent command`./compile emi_conv also created emiss_conv.exe utility. ###WPS compile###

2. WPS compilation was started with WPS tar file unzipping, using the command,

           gunzip WPSV3.4.1.TAR.gz
           tar -xf WPSV3.4.1.TAR
   

cd into it and running configure, the configure file was edited as per this and this, and run ./compile >& AWSWPSlog.compileA6, it created ungrib.exe but no metgrid.exe and geogrid.exe was created. In compilation log the error sated was this

            1 error 
            gfortran: error: /home/ubuntu/wrf/WRFV3/frame/    module_driver_constants.o: No such file or directory
            gfortran: error: /home/ubuntu/wrf/WRFV3/frame/module_machine.o: No such file or directory

1. This error was searched and get into this forum post in WRF forum. Indicating problem lies in WRF compilation itself. There is a error in wrf compilation which reads like below

           USE module_driver_constants
      1
           Fatal Error: Can't open module file                 'module_driver_constants.mod' for reading at (1):                 No such file or directory
   

The error showing in the WPS related to this error. It would be a problem with a single core in AWS free tier, but serial compilation also ends with a similar mistake. Unable to find a proper reason for this error, stopped the compilation in AWS temporarily.
1. This forum post worth noting that the usage of cores has limitations in WRF, which could significantly influence simulation output.