syrusakbary / promise

Ultra-performant Promise implementation in Python
MIT License
362 stars 76 forks source link

[Question, Enhancement] Why must dataloaders batch return a List? #39

Open japrogramer opened 7 years ago

japrogramer commented 7 years ago

https://github.com/syrusakbary/promise/blob/5177e5376676e10bbcd67970eed6c858c5d9bd4d/promise/dataloader.py#L31

Hello I know that bat_load_fn receives a list of keys as an argument and expect a list of the same length returned. My problem with this design is if call my batch method, from which i am getting my data and i receive an out of order result for some keys and not all keys I have to than sort and fill in values for keys not returned at the correct index.

class UserDataLoader(DataLoader):
    def batch_load_fn(self, keys):
        # Here we return a promise that will result on the
        # corresponding user for each key in keys
        def process_res(res):
            res_by_uuid = {str(row.uuid): row for row in res}
            return [res_by_uuid.get(key, None) for key in keys]
        return Promise.resolve(UserType.get_queryset().order_by().filter(uuid__in=keys)).\
            then(lambda res: res if len(res) is len(keys) else process_res(res))

So than I propose that the return value be changed to a dictionary of the form {key: value ...} with this no longer does the user need to worry about ordering, or missing data for keys because values for keys can be retrieved with

somedict.get(key, None)