DiamondLightSource / Opt-ID

Code for the Optimisation of ID's using Python and Opt-AI
Apache License 2.0
7 stars 6 forks source link

consider the way to run OptAI on the cluster, using qsub #17

Closed markbasham closed 10 years ago

markbasham commented 10 years ago

The way the system will run on the cluster is using the file system as a cache,

A node will be given a list of genome filenames, target fitness, lookup table location, and total number of mutations,

it will then do the following for each genome in filelist: load file, and look at fitness comparison with target fitness calculate number of children and number of mutations (This should basically multiply together to give the same number to make the process scale across the cluster nicely, given the previous ideas about mutation only calculations) create the new offspring, and calculate their fitneses and save the outputs into the next epoch directory

the main script will then load back in the data, and make use of this to specify the next epoch.

The filename of each genome, should probably have a unique ID plus the fitness and age in it to avoid having to load the file.

markbasham commented 10 years ago

'%010.8e%03i%s' %(23.5, 4, binascii.hexlify(os.urandom(8)))

Gives a good random value, which can be loaded and split up easily

markbasham commented 10 years ago

This is now sorted out and running nicely, seems to give good results so far on the simple test system