spotify / luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Apache License 2.0
17.92k stars 2.4k forks source link

ImportError: cannot import name 'MockFile' from 'luigi.mock' #3201

Open AbdAlRahman-Odeh-99 opened 2 years ago

AbdAlRahman-Odeh-99 commented 2 years ago

Greetings, I have a luigi task that doesn't return any output file, so I want to use MockFile instead to get it done. That being said, it gives me an error when trying to import it. The code:

import luigi
from luigi.mock import MockFile
import subprocess

lass PullImages(luigi.Task):
    task_namespace = 'CMS-Analysis'
    #task_complete = False
    def output(self):
        return MockFile("PullImages", mirror_on_stderr=True)

    def run(self):
        bashCommand = pullCommand()
        process = subprocess.Popen(bashCommand, shell=True, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
        output, error = process.communicate()
        print("The command is: \n",bashCommand)
        print("The output is: \n",output.decode())
        print("Return Code:", process.returncode)
        if process.returncode and error:
            print("The error is: \n",error.decode())
        else:
            self.output().value = True

The error message:

Traceback (most recent call last):
  File "cms-analysis.py", line 2, in <module>
    from luigi.mock import MockFile
ImportError: cannot import name 'MockFile' from 'luigi.mock' (/home/abd/.local/lib/python3.8/site-packages/luigi/mock.py)
abhinab-choudhury commented 1 year ago

Try replacing from luigi.mock import MockFile to from luigi.mock import MockFileSystem