errors in the import of exiobase hybrid units (3.3.x)

[mfastudillo]

If I try to import exiobase :

from bw2io.importers.exiobase3_hiot import Exiobase33Importer
assert path_to_exiobase.exists()
ei = Exiobase33Importer(path_to_exiobase)
bw.projects.set_current('Ilikephysicalunits')
ei.write_database()

I get a typerror

versions :

1. bw2data : 4.0.dev1
2. bw2io : 0.9.dev1

it seems the write method in the IOTablebackend has changed and crashes the Exiobase33Importer. Ideas of how to modify the arguments given to write to avoid this error ? perhaps adding the exchanges to the datasets ?

As a side note, the monetary version and the one with hybrid units are done by different institutions, and a different formart (thus different importers) are almost unavoidable. I'd change the importer name to reflect that.

[mfastudillo]

I tried to rewrite the write_database method for the exiobase33Importer but I am doing something wrong. The IOTableBackend has a couple of methods to write the data, write and write_exchanges. If I understood correctly the exchanges are stored as a processed array directly for performance reasons.

write_exchanges has as arguments technosphere and biosphere, with the format (row id, col id, value, flip) .. I tried to generate some old style biosphere and activity dicts, to change the keys for integers but it does not work. :-/

class ExiobaseHybridUnitsImporter(Exiobase33Importer):
    """modified version"""

    # very slow option
    def colrow_indices(self):
        """from an iterable of exchanges returns two
        dicts with the cols and rows corresponding to
        different products and their corresponding index
        on the A and B matrices. """
        col_dict = {}
        row_dict_B = {}
        col_i = 0
        row_i_B = 0

        for i,exchange in enumerate(self.exchanges):

            if exchange['output'] not in col_dict:
                col_dict[exchange['output']] = col_i
                col_i += 1

            if (exchange['type']=='biosphere') and (exchange['input'] not in row_dict_B):
                row_dict_B[exchange['input']] = row_i_B
                row_i_B += 1

        return(col_dict,row_dict_B)

    def write_database(self):
        mrio = IOTableBackend(self.db_name)
        mrio.write(self.datasets)

        activity_dict,biosphere_dict = self.colrow_indices()
        #write exchanges as processed arrays

        # generator expression of the form: (row id, col id, value, flip) 
        t_gen = ((activity_dict[ex['input']],activity_dict[ex['output']],ex['amount'],True) for ex in ei.exchanges if ex['type']=='technosphere')
        b_gen = ((biosphere_dict[ex['input']],activity_dict[ex['output']],ex['amount'],False) for ex in ei.exchanges if ex['type']=='biosphere')

        mrio.write_exchanges(technosphere = t_gen,
                             biosphere = b_gen,
                             dependents = [] )