Case: InMemory DB refuses to update after Making changes to model

Okafor-Ifeanyi commented 7 months ago

Versions

NodeJS: 18.18.0
mongodb-memory-server-*: 9.1.6
mongodb(the binary version): 5.9.0
mongodb(the js package): 5.9.0
mongoose: 7.6.3
system: Windows

package: mongo-memory-server

What is your question?

How do I get rid of the memory storage on mongodb-memory-server, cause it doesn't update the new edits on my model, I would like to scrape the dep out so the next time i install it it can reflect my changes and work perfectly

Case :

I am using mongodb-memory-server to run test on my project (school portal), developed with Typescript and Passport local strategy for Authentication. I initially had a mismatch of naming a model "session" which hand an entanglement with my cookie "session", So each time a cookie session is created, it tries to store the cookie session in the session model but like I said, I used that model to store a school session which I named "session". As expected this threw an error cause I had a unique constraint that failed each time the cookies "session" wanted to save in the session model.

cookie session => session model
school session => session model

unique_constraint_image Unique constraint highlighted > cause of error

My Solution:

The error hit me first while I was documenting my API on Postman (Error existed on both Postman and test), I then made an edit to the code cleared my session model and renamed my school session to "school_session" image below

fix-on-session-model Fix on Model

Instant fix on Postman and now passport auth goes smoothly.

cookie session =>session model
school session => school_session model

Issue:

Mongodb-memory-server doesn't take note of this change and is still throwing the same error

duplicate-key-error

This error occurs when the login route tries to pass the cookie session into the session model table and finds out that the session model table has a unique constraint on the name and semester which has already been used by the previous login session which had a null on the name and semester now the new login has the same null on name and semester and the error goes off.

All this shouldn't happen cause that model has been changed from Session Model to School_session Model.

I have tried:

Uninstalling Mongodb-memory-server
Dropping the database with await mongoose.connection.dropDatabase();
Deleting the temp files saved as mongo-mem
Suprisingly Restarting my System

Nothing works Please Help!!! this is like the 6th day

My Repo

Okafor-Ifeanyi commented 7 months ago

This is more of an Issue than a question please help. @hasezoey

hasezoey commented 7 months ago

mongodb-memory-server uses mongodb binaries directly, there is no dependency to clean out. also note that unless you are explicitly setting some database-path, all the data is only retained for the current run of the instance (if ephermeralForTest is used (which is the default below mongodb 7.0 here), almost no data is saved to disk, if wiredTiger is used, data is saved, but to a temporary (random generated) directory)

guessing from this you either:

have MMS running on some kind of watch but did not restart it (or clear it) to reset the database
your code as a issue which still results in the same usage as before
or you are using a custom database path, in which case you need to either dropDatabase it or delete the path

PS: mongodb(the binary version): 5.9.0 such a binary version does not exist

Okafor-Ifeanyi commented 7 months ago

Thank you for the response,

This is the test DB am connecting to..

I can't seem to place my hand on where exactly the error is coming from but current I have found a temporary solution of manually dropping my session Model on my atlas collection before running the test for some reason that works perfectly well.

Not to sound intrusive but could you please review my project I would appreciate any Critcism or design Modification suggestion you give on making it better, The repo is at the bottom of the initial question.

PS: mongodb(the binary version): 6.0.13 This is the correct version from my research

hasezoey commented 7 months ago

i just tried your repo, and it seems like you use env DATABASE_URI in src/middlewares/session.middleware.ts for your MongoStore and it never gets set to the testing database that gets spun up, so you always seem to hit your actual DB instead of the testing one; i got it fixed by moving the getting of the env to inside the function and setting the env variable in the test instead.

aside from that, i get a whole bunch of TCPWRAP errors (open handles), but the tests themself pass.

note that i used the test/admin branch like the url directed me to.

Okafor-Ifeanyi commented 6 months ago

Thank you so much for pointing out that error, that truly clarifies things for me

I have been trying to execute the fix you mentioned but with what I have implemented I get 500 on all my endpoints.

Added the mongoUrl to the function as shown above

Added the test db to session in image above

Please could you throw more light on getting of the env to inside the function and setting the env variable in the test instead. ??

Thank you once again.

hasezoey commented 6 months ago

here are the changes i did to get it working:

diff --git a/src/__tests__/admin.test.ts b/src/__tests__/admin.test.ts
index c090ef6..0254921 100644
--- a/src/__tests__/admin.test.ts
+++ b/src/__tests__/admin.test.ts
@@ -19,6 +19,7 @@ describe("Admin", () => {

   beforeAll(async () => {
     const mongoServer = await MongoMemoryServer.create();
+    process.env.DATABASE_URI = mongoServer.getUri();
     await mongoose.connect(mongoServer.getUri());

     // Create Admin User
diff --git a/src/middlewares/session.middleware.ts b/src/middlewares/session.middleware.ts
index 8cde955..fc91069 100644
--- a/src/middlewares/session.middleware.ts
+++ b/src/middlewares/session.middleware.ts
@@ -1,10 +1,11 @@
 import { NextFunction, Request, Response } from "express";
 import session from "express-session";
 import MongoStore from "connect-mongo";
-import { DATABASE_URI, MAXAGE, SECRET } from "../configs/constants.config";
-const mongoUrl = DATABASE_URI;
+import { MAXAGE, SECRET } from "../configs/constants.config";
+// const mongoUrl = DATABASE_URI;

 const sessionMiddleware = (req: Request, res: Response, next: NextFunction) => {
+  const mongoUrl = process.env.DATABASE_URI;
   return session({
     secret: SECRET,
     resave: false,

Note: sorry, this message somehow got posted before i was finished writing

Added the mongoUrl to the function as shown above

you can do that, be sure that you dont initialize it twice then

Please could you throw more light on getting of the env to inside the function and setting the env variable in the test instead. ??

if you want to continue using your configs import, then you will have to need to make them dynamic (so that on access they will fetch the correct one, instead of on module load)

Okafor-Ifeanyi commented 6 months ago

Just made the changes and the code works perfectly now, My Tests got exponentially faster once I started using the test db on Auth Session

Now all I need to do is get the TCPWRAPerror fixed

If you're in need of a Backend Intern or Developer to Hire I'm open to work.

Thank you for the help I appreciate.

typegoose / mongodb-memory-server