Generate Models from existing database

[draew6]

Is it possible to generate models from existing database without writing all models myself? Thanks

[tarsil]

That was actually discussed internally @draew6 but it was never implemented because it was not yet requested but it could be something to be added.

What if you put an example of what to expect and we can add to the roadmap?

[draew6]

I dont know what to expect as I am new to this topic but maybe this can help you somehow: https://github.com/agronholm/sqlacodegen which can generate declarative tables, dataclasses and sqlmodels(which is sqlalchemy+pydantic too) and maybe somehow transform it to Edgy models?

[tarsil]

Hmm, interesting actually. Let us see what we can do on our side. Is this blocking you? Is Edgy working ok for you so far?

[draew6]

@tarsil It's not blocking me. Actually I am at stage 0 and that's picking ORM and Edgy looks best in python ecosystem. But I also like drizzle and few others. Do not let it to change your priorities. Good luck with edgy ;)

[tarsil]

@draew6 by all means no worries at all and we can consider that.

Those are actually great options too do you and of course I hope you stick with Edgy ahaah

[rolandf]

A kind of inspectdb command would be really nice like in Django. If it was available I would jump into Esmerald/Edgy right away.

[tarsil]

@rolandf the good news is that, that was already in the roadmap. Tomorrow I will be starting to have a look and development so soon this should be out 🙂

[tarsil]

@rolandf currently running tests and it looks promising. The type of generated models will be of type edgy.ReflectModel (it's in the docs). This way you can do whatever you want after and this doesn't manage migrations (which is a safe side). The equivalent of the managed True of Django 🙂

[rolandf]

I tried:

env/bin/edgy inspectdb --database sqlite:///database.sqlite

I expected that to inspect database.sqlite

Got the following error:

/dev/QtPython66$ env/bin/edgy inspectdb --database sqlite:///database.sqlite 2023-11-04 18:35:00.556 | INFO | edgy.cli.operations.inspectdb:inspect_db:103 - Registry not found in the application. Using credentials... Logging in without authentication. Traceback (most recent call last): File "env/bin/edgy", line 33, in sys.exit(load_entry_point('edgy', 'console_scripts', 'edgy')()) File "/edgy/edgy/main.py", line 5, in run_cli edgy_cli() File "/env/lib/python3.8/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/env/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/edgy/edgy/cli/cli.py", line 76, in invoke return super().invoke(ctx) File "/env/lib/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/env/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/env/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/env/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, *kwargs) File "/edgy/edgy/cli/cli.py", line 53, in wrapped return func(args, kwargs) File "/edgy/edgy/cli/operations/inspectdb.py", line 115, in inspect_db metadata = execsync(reflect)(engine=engine, metadata=metadata) File "/edgy/edgy/core/sync.py", line 19, in wrapper return anyio.run(partial_func) File "/env/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 68, in run return asynclib.run(func, *args, backend_options) File "/env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 204, in run return native_run(wrapper(), debug=debug) File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete return future.result() File "/env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 199, in wrapper return await func(args) File "/edgy/edgy/cli/operations/inspectdb.py", line 173, in reflect async with engine.connect() as connection: File "/env/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/base.py", line 125, in aenter return await self.start(is_ctxmanager=True) File "/env/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py", line 270, in start await greenlet_spawn(self.sync_engine.connect) File "/env/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn result = context.throw(sys.exc_info()) File "/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3268, in connect return self._connection_cls(self) File "/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 145, in init self._dbapi_connection = engine.raw_connection() File "/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3292, in raw_connection return self.pool.connect() File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 452, in connect return _ConnectionFairy._checkout(self) File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1269, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 716, in checkout rec = pool._do_get() File "/env/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 170, in _do_get self._dec_overflow() File "/env/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 146, in exit raise exc_value.with_traceback(exc_tb) File "/env/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 167, in _do_get return self._create_connection() File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection return _ConnectionRecord(self) File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 678, in init self.connect() File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 903, in connect pool.logger.debug("Error on connect(): %s", e) File "/env/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 146, in exit raise exc_value.with_traceback(exc_tb) File "/env/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 898, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/env/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 637, in connect return dialect.connect(*cargs, *cparams) File "/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 616, in connect return self.loaded_dbapi.connect(cargs, cparams) File "/env/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 936, in connect await_only(creator_fn(*arg, kw)), File "/env/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only return current.driver.switch(awaitable) # type: ignore[no-any-return] File "/env/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn value = await result File "/env/lib/python3.8/site-packages/asyncpg/connection.py", line 2114, in connect return await connect_utils._connect( File "/env/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 1011, in _connect raise last_error or exceptions.TargetServerAttributeNotMatched( File "/env/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 982, in _connect conn = await _connect_addr( File "/env/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 808, in _connect_addr return await connect_addr(params, timeout, True, *args) File "/env/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 860, in connect_addr tr, pr = await compat.wait_for(connector, timeout=timeout) File "/env/lib/python3.8/site-packages/asyncpg/compat.py", line 60, in wait_for return await asyncio.wait_for(fut, timeout) File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for return fut.result() File "/env/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 719, in _create_ssl_connection tr, pr = await loop.create_connection( File "/usr/lib/python3.8/asyncio/base_events.py", line 986, in create_connection infos = await self._ensure_resolved( File "/usr/lib/python3.8/asyncio/base_events.py", line 1365, in _ensure_resolved return await loop.getaddrinfo(host, port, family=family, type=type, File "/usr/lib/python3.8/asyncio/base_events.py", line 825, in getaddrinfo return await self.run_in_executor( File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, self.kwargs) File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

[tarsil]

@rolandf the inspectdb is still under development. I was just telling you what is coming 🙂

[tarsil]

@rolandf the inspectdb is still under development. I was just telling you what is coming 🙂

[rolandf]

No problem, I was not sure if I tried it a bit too early or if I made a mistake in the way I called the command. I'm trying it with sqlite db. If you need a beta tester I'm available.

[tarsil]

Well, it's too early because it's on a branch but I so appreciate the initial tests. I still need to add SQLite sincr the schema and connection string is parsed differently from the rest. At the moment edgy is napping stuff. Hopefully next week a fully working version will be out but the way you ran the command will be the way of doing it, at least for the current design. Thank you for the feedback

[tarsil]

Also once I have a working version I will ask you to checkout it and test it @rolandf . Is that ok with you?

[rolandf]

Cool, I save the command for later then. Yes, you are very welcome to reach out to me for testing. I used ChatGPT to convert the pyproject.toml file into a setup.py file so I can "pip install -e ." from the branch checkout.

[tarsil]

Hmm interesting because I always use the pip install -e . With pyproject without any problem @rolandf

[tarsil]

Also @rolandf i tend to have people like yourself involved in the development and testing process. Which is great to make sure everything is delivered properly. This is how the multi tenancy support came to be. Stay tuned and during the week you will have something to play around. Promise

[tarsil]

@rolandf i have moved this conversation to https://github.com/tarsil/edgy/discussions/42