Circular reference works on prod, not locally (help needed, not a bug)

[jansensan]

Question

I run Arts & Médias, a directory of discoveries and references to Media Art critical content. For a while now, I have been updating the content on prod, then pulling a copy of the prod database and uploads to a local machine for development on the site.

Recently though, it seems that there might be pieces that reference each other (circular references). I can't get the error to show anymore, but that was once mentioned in the console errors. That is not impossible either

Now, when I run the site locally, this line is looping infinitely in the console:

[###############|     ] 9451 of 12416 (76.12%) ETA: 2s 1142/sec

While not an ApostropheCMS issue directly, it is indirectly related, because seemingly Apos cannot start the project if circular references exist.

Is there a way to find out which piece could refer to another in that way, without going through the dozen pieces one at a time? (Keep in mind that I am not a backend/database developer, so any help may need explicit explanations, no "obviously" this or "easy" that.)

Note that this works perfectly fine remotely.

Details

Version of Node.js: v16.19.0 (with nvm)

Server Operating System: Local machine (where issue occurs): macos Catalina 10.15.7

Prod machine (where things work fine): Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-197-generic x86_64) v16.19.0 (with nvm)

I'll gladly share more details if needed to help me figure this out.

[boutell]

Hi Mat,

I have never seen this issue before. Can you share more of the console output that makes you suspect the infinite startup time locally is due to a circular reference?

On Thu, Jan 12, 2023 at 8:18 AM Tom Boutell @.***> wrote:

Hi Mat,

While circular references are bad for site performance Apostrophe contains mechanisms to shut that down before it becomes an infinite loop (this works better in A3, but it also works in A2).

So I suspect this might not be the actual issue. But it would help to see more of the console output you showed before.

But in the long run it might not be possible to really diagnose this without access to the database and code in question. It is not a behavior I have ever seen before.

On Wed, Jan 11, 2023 at 4:28 PM Mat Janson Blanchet < @.***> wrote:

Question

I run Arts & Médias https://arts-et-medias.net/, a directory of discoveries and references to Media Art critical content. For a while now, I have been updating the content on prod, then pulling a copy of the prod database and uploads to a local machine for development on the site.

Recently though, it seems that there might be pieces that reference each other (circular references). I can't get the error to show anymore, but that was once mentioned in the console errors. That is not impossible either

Now, when I run the site locally, this line is looping infinitely in the console:

[###############| ] 9451 of 12416 (76.12%) ETA: 2s 1142/sec

While not an ApostropheCMS issue directly, it is indirectly related, because seemingly Apos cannot start the project if circular references exist.

Is there a way to find out which piece could refer to another in that way, without going through the dozen pieces one at a time? (Keep in mind that I am not a backend/database developer, so any help may need explicit explanations, no "obviously" this or "easy" that.)

Note that this works perfectly fine remotely, on a Details

Version of Node.js: v16.19.0 (with nvm)

Server Operating System: Local machine (where issue occurs): macos Catalina 10.15.7

Prod machine (where things work fine): Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-197-generic x86_64) v16.19.0 (with nvm)

I'll gladly share more details if needed to help me figure this out.

— Reply to this email directly, view it on GitHub https://github.com/apostrophecms/apostrophe/issues/3996, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH27POUT3CQGLVIG4WBYLWR4QZFANCNFSM6AAAAAATYQIXAI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[jansensan]

Hello Tom, thanks for following up.

I unfortunately cannot get the specific error anymore. When I did, I found that an update on mongodb (3.7.3 I believe, looking at my commit history) had changed something, then I updated the dependencies a bit, and the error went away. However, I do not believe I have fix the content that could have been the cause.

My investigation may also be incorrect.

Here are two screen captures of what currently happens (not much more, sorry):

NodeJS v14

https://user-images.githubusercontent.com/1410355/212130141-39d6c809-8579-4cc9-bebf-013953ac6a3f.mp4

NodeJS v16

https://user-images.githubusercontent.com/1410355/212130319-dad99b06-dfe3-4e3c-9ffc-391e5f8f0acc.mp4

[jansensan]

I reverted back a few changes and found it:

(node:6981) Warning: Accessing non-existent property 'MongoError' of module exports inside circular dependency

Here is the diff between the package.json; the one on the right shows the error above, the one on the left does not. Both loop infinitely though.

Like I said in the description, it might not be Apos' fault, but I thought I'd ask anyway, in case you have an idea.

[boutell]

Not sure. Glad all is well after your "npm update".

On Thu, Jan 12, 2023 at 12:02 PM Mat Janson Blanchet < @.***> wrote:

I reverted back a few changes and found it:

(node:6981) Warning: Accessing non-existent property 'MongoError' of module exports inside circular dependency

Here is the diff between the package.json; the one on the right shows the error above, the one on the left does not. Both loop infinitely though.

[image: image] https://user-images.githubusercontent.com/1410355/212131880-b9696cb4-39e1-4507-9809-975aeb711e8b.png

Like I said in the description, it might not be Apos' fault, but I thought I'd ask anyway, in case you have an idea.

— Reply to this email directly, view it on GitHub https://github.com/apostrophecms/apostrophe/issues/3996#issuecomment-1380728117, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH27LUFL2HHNZ22DBFSYTWSA2JTANCNFSM6AAAAAATYQIXAI . You are receiving this because you commented.Message ID: @.***>

--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[jansensan]

Well no, to be clear, not all is well after the update, only the circular error is gone. I still cannot launch my Apos project locally. As can be seen from the video screen captures above, the terminal/console loops infinitely, going through all the pieces for some reason, with no details.

Edit: I am not being snarky or attacking, I just want to ensure clarity :)

[boutell]

Oh no worries. Clarity is great.

Is there a way you can share a project where others can reproduce this?

[jansensan]

• The project source is public, so you can get a copy of the code here: https://github.com/emjibay/arts-et-medias/tree/develop

• For the uploads and the db, I uploaded a copy to WeTransfer: https://we.tl/t-7nGrb2o0AI. I think they offer a single download with a link? Let me know if you did not get it.

• It may not be necessary to tell you, but that's how I'd populate the local DB with the one provided in the zip: mongorestore --gzip --archive=aem-db-20230111-1335-prod.tgz --drop

• You probably will need an .env file, my local one contains this:

  APOS_ENV='development'
  HOST='0.0.0.0'
  PORT='3000'

• The project uses Yarn. Once you have the dependencies installed, I would run yarn start or yarn dev depending on the need.

I apologize in advance for any optimization lacks you may find, my specialty is user interaction and interfaces, not backend, services, or optimization 😅

[BoDonkey]

Hi @jansensan, Your site is beautiful! I have tried without luck to get your error to repeat locally on my machine. With yarn start I'm getting notifications about enableAltField and that the sponsor doc type has unarranged fields, but everything is starting up normally. I've tried pining the dependencies in the package.json to the ones you identified above, and I've tried multiple versions of Node. My best guess is a corrupt local database. Not sure what you have tried there. This error was seen by a number of others in v3.6.4 of the MongoDB driver for node. I don't think this should be a problem in your local project, but maybe check the driver version. Sorry I haven't been a little more help. Somebody else on the team may be able to jump in and comment. Bob

[jansensan]

@BoDonkey Thanks for the praise 🙏

I noticed the enableAltField notifications, but I had handled that manually a long time ago.

I had not considered a corrupt database, however after dropping the whole local version and populating a new one with the remote content, still no success. Databases are completely outside of the scope of my knowledge though...

[jansensan]

@boutell @BoDonkey So following Robert's suggestion, I tried (but failed) to investigate the database. Instead, I tried to completely uninstall brew and mongo. After a few swears and repeated commands, it actually worked, with no changes in the database itself.

Thanks to both of you for trying, even though all this was not actually an issue with Apos' code but rather my machine

[boutell]

All's well that ends well!