Using custom datareader with pabot tests are not executed in parallel

[Vance-Turner]

Adding to this issue tracker as well since I'm not sure which library is causing the issue. Issue on pabot https://github.com/mkorpela/pabot/issues/262

pabot is not parallelizing tests with DataDriver when using a custom data reader. The tests are all created by DataDriver and are successfully executed but sequentially. I put together a simple test project. Directory structure:

• resources
  • py3
  • test
    • datareader
    • customreader.py
    • simplemodule.py
• robot
  • cases.xlsx
  • test.robot

Contents of customreader.py (Direct copy of the XLSX reader in DataDriver).

from test.datareader.simplemodule import aSillyFunction
from DataDriver.AbstractReaderClass import AbstractReaderClass
import pandas as pd
import numpy as np

class customreader(AbstractReaderClass):

    def get_data_from_source(self):
        data_frame = pd.read_excel(self.file, sheet_name=self.sheet_name, dtype=str).replace(np.nan, '', regex=True)
        self._analyse_header(list(data_frame))
        for row in data_frame.values.tolist():
            self._read_data_from_table(row)
        return self.data_table

Contents of simplemodule.py

def aSillyFunction(x):
    return 2*x

Contents of test.robot

*** Settings ***
Library       DataDriver  file=cases.xlsx  reader_class=../resources/py3/test/datareader/customreader.py

Test Template  Run Test

*** Test Cases ***
Dummy Run Test  ${x1}  ${x2}   ${1}  ${1}

*** Keywords ***
Run Test
    [Arguments]  ${x1}  ${x2}
    ${x1}  convert to number  ${x1}
    ${x2}  convert to number  ${x2}
    should be equal  ${x1}  ${x2}

Running with: pabot --processes 3 --testlevelsplit --pythonpath ../resources/py3 test.robot

Now, if we change the import in customreader.py from:

from test.datareader.simplemodule import aSillyFunction to from simplemodule import aSillyFunction pabot will parallelize the tests. Switching the import back though causes pabot to execute all tests sequentially again. Note that we need to remove .pabotsuitenames after running with the full module path import and switching.

[mkorpela]

As DataDriver - pabot binding is a custom thing I would start from here. Pabot assumes that DataDriver does few tricks and maybe this does not happen when using custom datareader.

[Snooz82]

That is funny, because everything which belongs to DYNAMICTESTS is in DataDriver class and not in the readers.

I will check this...

[Snooz82]

i really cant reproduce you situation. I think the import from test.datareader.simplemodule import aSillyFunction just do not work and DataDriver then runs into an exception, because the custom_reader can not been imported at all.

i would assume that this is not a bug in DataDriver, it is just a wrong import...

When this happens, because DataDriver does absolutely nothing, Pabot does not know that there are DYNAMICTESTs. so Pabot create a pabotsuitenames with just the Suite in it and not Test Cases. This leads to the situation, that on the next run, when pabotsuitenames already exists, it executes only the Suite and does not execute the tests in parallel.

BUT: it found a "bug": the import of custom readers can be a relative path, but it must be relative to the folder where robot is executed. This is somehow strange. It is now also possible to use a relative path to *.robot file. In your case it was the same, because you executed robot from the same directory.

it is available in 0.4.0b1 pip install -U --pre robotframework-datadriver

Ps: If you think this is not the case (with the wrong import) please upload a project where we can reproduce it. Because i wasn’t able to get this import running at all.

[Vance-Turner]

Thank-you. I have attached an example. TestPabotCustomDataReader.zip

There is something strange going on with the Python path though. I was testing the example which I uploaded changing the import back and forth; and I was getting the expected results. Just before I went to zip, I removed all the generated files and some hidden directories and then ran again: only the from simplemodule import aSillyFunction import will work now. Using the import from test.datareader.simplemodule import aSillyFunction is causing the test to not even execute due to the module test.datareader not being found on the Python path. The stack trace in log.html though clearly shows the absolute path to the py3 directory showing up. I sent the example to a colleague. The example runs for her but test cases do not get parallelized (as expected with the import).

[Snooz82]

i close it due to completely new implementation with Pabot will come soon