senwu / emmental

A deep learning framework for building multimodal multi-task learning systems.
https://emmental.readthedocs.io
MIT License
109 stars 18 forks source link

Fix NoneType issue in collate_fn when using python 3.8+ on mac #117

Closed senwu closed 2 years ago

senwu commented 2 years ago

Description of the problems or issues

On Mac with python 3.8+ multiprocessing uses spawn to start multiple process and it won't inherit from the origin process which causes Meta.config is None. This results emmental_collate_fn cannot get min_data_len and max_data_len from Meta.config.

Description of the proposed changes

Instead get min_data_len and max_data_len from Meta.config in Emmental_collate_fn, we set those two variables in the EmmentalDataLoader.

Test plan

Passes the existing test.

Checklist

codecov[bot] commented 2 years ago

Codecov Report

Merging #117 (b417339) into master (17b5075) will increase coverage by 0.01%. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #117      +/-   ##
==========================================
+ Coverage   92.12%   92.13%   +0.01%     
==========================================
  Files          40       40              
  Lines        2018     2021       +3     
  Branches      431      432       +1     
==========================================
+ Hits         1859     1862       +3     
  Misses         94       94              
  Partials       65       65              
Flag Coverage Δ
unittests 92.13% <100.00%> (+0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/emmental/data.py 91.74% <100.00%> (+0.23%) :arrow_up: