Generate new models from data-files

[cortner]

try to use QUIP data files, talk to Noam about NRLTB and to Lars about his TB models?

[cortner]

@hjchen1983 just assigned to you :);

but maybe focus on the online database instead of QUIP?

[jarvist]

I dusted off the NRL reader code: https://github.com/jarvist/TightBinding.jl/commit/146d86655ed08c8e1df1cdfc5e5d3016e8b75952

It seems to work, though for Ti.par, I had to delete a line in the parameter file, to get it to line up with the expected row numbers after readdlm in https://github.com/jarvist/TightBinding.jl/blob/146d86655ed08c8e1df1cdfc5e5d3016e8b75952/src/NRLTB_data2.jl#L7-L58 .

As the original NRL website (http://cst-www.nrl.navy.mil/bind/) is no longer there, this data was recovered from the Wayback machine (https://web.archive.org/web/20121003160812/http://cst-www.nrl.navy.mil/bind/). It might be an idea to do that for all the data, and put it in a separate nrl_data repository.

Generally the code could do with being restructured a bit. Perhaps NRL_data2.jl --> NRL_parameter_reader.jl ?

[jarvist]

Did the TBDataFiles module get anywhere, or is this a code hook for something that hasn't yet been written? https://github.com/cortner/TightBinding.jl/blob/177a98c8bc4d883c95249c93ce7efd89498ef808/src/NRLTB_data.jl#L7-L10

[jarvist]

re: QUIP data + package, these are at: https://github.com/libAtoms/QUIP/tree/public/share/Parameters

They appear to have some of the NRL parameters. All are in an XML format.

[cortner]

It might be an idea to do that for all the data, and put it in a separate nrl_data repository.

we've already done this :), though maybe not all of them. I'll have a look. But we made it private since we were worried about copyright, etc. Do you have any ideas about this? In some sense it would be great to just keep all these files in a public GitHub repository. I think the problem is that the authors want to sell their book.

[cortner]

Did the TBDataFiles module get anywhere, or is this a code hook for something that hasn't yet been written?

I admit that I have no memory of what was written / discussed. @hjchen1983 did you write some code to load the NRLTB data files?

[cortner]

Ok, I found the repository TBDataFiles. It now looks like we were planning to create such a repository but never actually did. But I made a note there:

There are no licenses on that website, which most likely means that we cannot publish these files, but it will be ok to use them within our own research group.

Anyhow - anything you want to do to create a repository of data files (if allowed) would be very useful for us too!

[jarvist]

Anyhow - anything you want to do to create a repository of data files (if allowed) would be very useful for us too!

I'll have a look into it + consider licensing issues.

I guess one possibility is have a script which populates the nrl_data folder by downloading from the WayBack machine. I'll have to check whether the WayBack machine is happy with robots.

[cortner]

Is it guaranteed that the data will be indefinitely available?

[jarvist]

I'm not sure. The original website can change their robots.txt, and I think the WayBack machine will respect that & stop showing the pages.

The original paper (https://doi.org/10.1103/PhysRevB.54.4519) states that "The tight-binding parameters used in this paper are available from..." and then suggests a AIP URL, and the nrl.navy.mil URL, or directly emailing Mehl. So it's pretty clear they wanted them to be available!

[cortner]

@hjchen1983 had contacted Mehl when we started with all this, and had the feeling he wasn't too keen to just give people the data, without buying their book. I think we should look into licensing issues, but even if there is an issue, it may still be acceptable to just download all the data-files and store them for personal use? Maybe in a private repository? (that was our original plan)

[jarvist]

OK; so it was pretty trivial to write a little shell script that downloads all the dereferenced species in NRL_data2.jl. The WayBack machine is very nice in that it recognises that wget is not a browser, and so doesn't put that banner across the top of the page, just giving you the pure data instead.

> cat scrape_params.sh 
for species in  "ag_par" "al_par" "al_par_t2g_eq_eg" "au_par" "ba_par"\
    "ca_par" "ca_par_315" "co_par" "cr_par" "cu_par" "fe_para_par"\
    "fe_ferro_par" "ga_par" "hf_par" "in_par" "ir_par" "mg_par"\
    "mo_par" "nb_par" "ni_par" "os_par" "pb_par" "pt_par" "re_par"\
    "rh_par" "ru_par" "sc_par" "sr_par" "ta_par" "tc_par" "Ti_par"\
    "ti_01_par" "v_par" "w_par" "y_par" "zr_par"
do
 wget "https://web.archive.org/web/20121003160812/http://cst-www.nrl.navy.mil/bind/${species}"
done

Next I will check by hand and see which species of the periodic table are missing.

I think it would be nice to clean up NRL_data & NRL_data2 - certainly I don't think the 2 as a suffix should be used to show that the routine is reading in parameters!

[cortner]

Fantastic - please feel 100% free to pursue this as you think is best

[jarvist]

OK; I think this is all available parameters.

# Quoted species are from NRL_data2.jl
# Further ones entered by hand.
for species in  "ag_par" "al_par" "al_par_t2g_eq_eg" "au_par" "ba_par"\
    "ca_par" "ca_par_315" "co_par" "cr_par" "cu_par" "fe_para_par"\
    "fe_ferro_par" "ga_par" "hf_par" "in_par" "ir_par" "mg_par"\
    "mo_par" "nb_par" "ni_par" "os_par" "pb_par" "pt_par" "re_par"\
    "rh_par" "ru_par" "sc_par" "sr_par" "ta_par" "tc_par" "Ti_par"\
    "ti_01_par" "v_par" "w_par" "y_par" "zr_par"\
    c_par c_par.105\
    si_par.125 si_par.spd si_par\
    ti_par ti_gga_par ti_par_01\
    mn_par\
    fe_par\
    co_para_par co_ferro_par\
    cu_par_99\
    ge_par.sp.125 ge_par.spd.125\
    pd_par.105 pd_par\
    sn_case1_par sn_case2_par
#Nb: ti_01_par above seems incorrect?
do
 wget "https://web.archive.org/web/20121003160812/http://cst-www.nrl.navy.mil/bind/${species}"
done

[jarvist]

Given the non-existence of TBDataFiles, do you think we should just merge NRL_data2.jl into NRL_data.jl & keep the simple parser in-house? I should also read how you are meant to do data files in a Julian way, the present seems a little coarse!

        cd(homedir())
        cd(".julia/v0.6/TightBinding/nrl_data") # Relatively universal!

[cortner]

Given the non-existence of TBDataFiles, do you think we should just merge NRL_data2.jl into NRL_data.jl & keep the simple parser in-house?

sounds good to me.

I should also read how you are meant to do data files in a Julian way, the present seems a little coarse!

maybe using build.jl ? The user can then call Pkg.build("TightBinding") and the files will be downloaded into a data directory. The nice way to do it is

julia> Pkg.dir("TightBinding")
"/Users/ortner/.julia/v0.6/TightBinding"

[cortner]

22