GenericCSVData class may use lambda functions which prevent it to be used in optimizations

[vladisld]

Community discussion:

https://community.backtrader.com/topic/2919/error-in-multi-core-optimization

Description:

using GenericCSVData class or any class inherited from it in strategy optimization may result in pickle error:

AttributeError: Can't pickle local object 'GenericCSVData.start.<locals>.<lambda>'

This happens in case the CSV file contains the timestemps in a datetime column. Looking at the code it appears that GenericCSVData is using lambdas for converting the timestamps to the UTC time. Unfortunately lambda functions could not be serialized by pickle package and thus can not be used during optimization with multiple CPUs.

def start(self):
        super(GenericCSVData, self).start()

        self._dtstr = False
        if isinstance(self.p.dtformat, string_types):
            self._dtstr = True
        elif isinstance(self.p.dtformat, integer_types):
            idt = int(self.p.dtformat)
            if idt == 1:
                self._dtconvert = lambda x: datetime.utcfromtimestamp(int(x))
            elif idt == 2:
                self._dtconvert = lambda x: datetime.utcfromtimestamp(float(x))

        else:  # assume callable
            self._dtconvert = self.p.dtformat

[mikeejazmines]

Hi! Was this confirmed to fix the issue?

[vladisld]

There were no PR submitted yet for this issue. The above commit from @viper7882 is in private fork and doesn't seems to address the whole issue.