Add DuckDB module

[31z4]

Hi all 👋 I'm attending Open Data Hack and building a project that gathers public data about Filecoin Saturn network and analyses these data to gain useful insights. I'd love to use Lilypad for the analysis ☺️

So in this PR I introduce the initial version of DuckDB 🦆 module that allows to run arbitrary SQL queries against IPFS/Filecoin data.

As discussed earlier on #bacalhau-lilypad Slack channel I publish this PR without throughout testing hoping for help from maintainers.

Although, the Docker image itself and the job spec was successfully tested with Bacalhau by running real SQL query against real data sitting on IPFS.

[jochasinga]

Hi @binocarlos @31z4 I know this is merged, but wondering if just by changing the two files in the commit a new module will just register and work. I try adding a small "whalesay" module myself by adding the two files and I got the following error after running lilypad run whalesay:v0.0.1 "hello whale module\!":

File "/usr/local/bin/modicum", line 33, in <module>
    sys.exit(load_entry_point('Modicum', 'console_scripts', 'modicum')())
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/modicum/cli.py", line 733, in runLilypadCLI
    exitcode = JC.postLilypadOffer(template, params)
  File "/app/modicum/JobCreator.py", line 453, in postLilypadOffer
    txHash = self.ethclient.transact(
  File "/app/modicum/EthereumClient.py", line 104, in transact
    return method.transact(params)
  File "/usr/local/lib/python3.10/dist-packages/web3/contract/contract.py", line 479, in transact
    return transact_with_contract_function(
  File "/usr/local/lib/python3.10/dist-packages/web3/contract/utils.py", line 172, in transact_with_contract_function
    txn_hash = w3.eth.send_transaction(transact_transaction)
  File "/usr/local/lib/python3.10/dist-packages/web3/eth/eth.py", line 362, in send_transaction
    return self._send_transaction(transaction)
  File "/usr/local/lib/python3.10/dist-packages/web3/module.py", line 68, in caller
    result = w3.manager.request_blocking(
  File "/usr/local/lib/python3.10/dist-packages/web3/manager.py", line 231, in request_blocking
    response = self._make_request(method, params)
  File "/usr/local/lib/python3.10/dist-packages/web3/manager.py", line 175, in _make_request
    return request_func(method, params)
  File "/usr/local/lib/python3.10/dist-packages/web3/middleware/signing.py", line 161, in middleware
    transaction = format_and_fill_tx(params[0])
  File "cytoolz/functoolz.pyx", line 518, in cytoolz.functoolz.Compose.__call__
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.10/dist-packages/web3/_utils/transactions.py", line 132, in fill_transaction_defaults
    default_val = default_getter(w3, transaction)
  File "/usr/local/lib/python3.10/dist-packages/web3/_utils/transactions.py", line 72, in <lambda>
    "gas": lambda w3, tx: w3.eth.estimate_gas(tx),
  File "/usr/local/lib/python3.10/dist-packages/web3/eth/eth.py", line 292, in estimate_gas
    return self._estimate_gas(transaction, block_identifier)
  File "/usr/local/lib/python3.10/dist-packages/web3/module.py", line 68, in caller
    result = w3.manager.request_blocking(
  File "/usr/local/lib/python3.10/dist-packages/web3/manager.py", line 232, in request_blocking
    return self.formatted_response(
  File "/usr/local/lib/python3.10/dist-packages/web3/manager.py", line 197, in formatted_response
    apply_error_formatters(error_formatters, response)
  File "/usr/local/lib/python3.10/dist-packages/web3/manager.py", line 73, in apply_error_formatters
    formatted_resp = pipe(response, error_formatters)
  File "cytoolz/functoolz.pyx", line 680, in cytoolz.functoolz.pipe
  File "cytoolz/functoolz.pyx", line 655, in cytoolz.functoolz.c_pipe
  File "/usr/local/lib/python3.10/dist-packages/web3/_utils/method_formatters.py", line 782, in raise_contract_logic_error_on_revert
    raise ContractLogicError(message, data=data)
web3.exceptions.ContractLogicError: execution reverted: Module not found

Looks like the new module isn't found in the contract code itself. In order to add a module, should the contract be modified and redeployed?

[binocarlos]

@jochasinga hey - thanks for your message and it's awesome you are creating new modules :+1:

The reason the above error is showing is we need to do a few things to deploy a new module - namely:

• add the module to Modules.py (seems like you've done that part)
• deploy the new code to the resource providers so they pick up on the module name
• add the module to the smart contract

It's the last step which is why you are seeing the error above - if you make a PR for the WhaleSay module - I will deploy it alongside the DuckDB one early next week :+1: