models

[TingTingShao]

Hi,

The job:

shell:
    """
    pycistopic topic_modeling mallet \
        -i {input.obj_pkl} \
        -o {output.topic_pkl} \
        -t 2 5 10 20 30 35 40 45 50 55 60 65 70 80 90\
        -p 16 \
        -b {config[mallet]} \
        -T {config[tmp_dir]} \
        -k True       
    """

After submitting the job to the server, I realized that the each model will take 4-5 hours to complete, but I set up 36hours for this job.

I know this job will fail eventually, could I please ask if there is a way, for example

The job stopped after finishing model with 45 topics, is it possible for me to run again with only 50 55 60 65 70 80 90,

    pycistopic topic_modeling mallet \
        -i {input.obj_pkl} \
        -o {output.topic_pkl} \
        -t 50 55 60 65 70 80 90\
        -p 16 \
        -b {config[mallet]} \
        -T {config[tmp_dir]} \
        -k True 

Assume the last run is successful, and it generates one .pkl file? how to add the models from previous run to the .pkl file? I tried with an example with append, but didn't look very right.

Thanks! tingting

[TingTingShao]

Thread https://github.com/aertslab/pycisTopic/issues/106 solved my first question.

[yojetsharma]

Hi, any idea how i can import modules like pycistopic.diff_features while running it as a job? pycistopic module gets imported when i submit the job on SGE but not the specific modules like diff_features, normalize_scores. These are normally imported in jupyter or python though.