Schema Migration Overriding exist Indexes

badboy-tian commented 2 months ago

New Issue Checklist

Report security issues confidentially.
Any contribution is under this license.
Before posting search existing issues.

Issue Description

Schema Migration Overriding exist Indexes

Steps to reproduce

1.create some index in mongodb tools 2.define ExportTool class in schema

export const ExportTotal = {
    className: "ExportTotal",
    fields: {
        myid: {type: "Number", defaultValue: 0},
        money: {type: "Number", defaultValue: 0},
        type: {type: "String"},
        user: {type: "Pointer", targetClass: "_User"},
    },

    classLevelPermissions: {
        find: {},
        count: {},
        get: {},
        update: {},
        create: {},
        delete: {},
        protectedFields: {},
    },
}

3.config the parse-server

schema: {
        definitions:[ExportTool],
        lockSchemas: false,
        strict: true,
        recreateModifiedFields: true,
        deleteExtraFields: true,
    },

Actual Outcome

when parse server running, the exist index has dispear

Expected Outcome

when parse server running, the exist index shoud not dispear

Environment

Server

Parse Server version: 7.0.0
Operating system: macos
Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): local

Database

System (MongoDB or Postgres): mongodb
Database version: 7.0.8
Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): local

Client

SDK (iOS, Android, JavaScript, PHP, Unity, etc): no
SDK version: no

Logs

info: 11:34:54 -> Node app is running on port: 1337
[TS] info: Running Migrations
[TS] info: Running Migrations Completed

parse-github-assistant[bot] commented 2 months ago

Thanks for opening this issue!

🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

mtrezza commented 2 months ago

What is the expected behavior according to the defined schema docs? I couldn't any specific mention of this in the docs.

badboy-tian commented 2 months ago

@mtrezza I hope the "Schema Migration" doesn't overwrite the Index I created with the mongodb tool!

badboy-tian commented 2 months ago

@mtrezza because the follow can not create unique index

indexes: {
    tagsIndex: { 
                          tags: 1 ,
                          // unique: true ?
                      }

so i must create unique index in mongodb shell or mongodb tool. then i run the "the defined schema" in parse-server, when log

info: 11:34:54 -> Node app is running on port: 1337
[TS] info: Running Migrations
[TS] info: Running Migrations Completed

the unique index created by mongodb shell was dispear.... Overriding ?

mtrezza commented 2 months ago

Some behavior is not explicitly described for indexes in the docs. For example if you set strict: true, then I would assume that indexes that aren't defined in the schema would be deleted, just like classes, fields, CPLs. In see in your case you have also set this option.

Did you take a look at the code to investigate what the current behavior is regarding indexes, so we can update the docs with the status quo? Then we can think about what we want to change.

badboy-tian commented 2 months ago

@mtrezza yes, when set strict: false the exist indexes was not overrided !

but because the index define in the define schema can not support create unique index, so we have to create unique index in mongodb tools. very inconvenient.

is there any way to supprot this?

indexes: {
   tag_index: {
          tag: 1, 
          unique: true, 
          background: true,
          sparse: true
    }

}

thank you

rgunindi commented 2 months ago

Some behavior is not explicitly described for indexes in the docs. For example if you set strict: true, then I would assume that indexes that aren't defined in the schema would be deleted, just like classes, fields, CPLs. In see in your case you have also set this option.

Did you take a look at the code to investigate what the current behavior is regarding indexes, so we can update the docs with the status quo? Then we can think about what we want to change.

And also for strict true or false whatever result the same[also with another options]. I debugged with changing options but the result will be the same: overrides indexes. @mtrezza

mtrezza commented 2 months ago

when set strict: false the exist indexes was not overrided !

for strict true or false whatever result the same ... overrides indexes

Which one is true?

badboy-tian commented 2 months ago

perhaps there is a unknow bug. i set strict: false, the index also overrided , i do not know why

badboy-tian commented 2 months ago

https://github.com/parse-community/parse-server/assets/2835129/d5c55474-246a-4199-b72e-409aba01098a

@mtrezza @rgunindi please see the screen record, set strict: false, also override the exist index

badboy-tian commented 2 months ago

@mtrezza @rgunindi

rgunindi commented 2 months ago

https://github.com/parse-community/parse-server/assets/2835129/d5c55474-246a-4199-b72e-409aba01098a

@mtrezza @rgunindi please see the screen record, set strict: false, also override the exist index

Yes ı said, for strict true or false whatever result the same ... overrides indexes

badboy-tian commented 2 months ago

@rgunindi yes, you are right

badboy-tian commented 1 month ago

any update ?

rgunindi commented 1 month ago

Don't expect early action 😁 Will take so long time

badboy-tian commented 1 month ago

@rgunindi yes. i have to create schema use api, create index use mongodb shell

badboy-tian commented 1 month ago

I find anthor way to merge schema, and the indexs can not override @rgunindi example:

export const IMessageReply = {
    className: "IMessageReply",
    fields: {
        fromUser: {type: "Pointer", targetClass: "_User"},
        message: {type: "Pointer", targetClass: "IMessage"},
        content: {type: "String"},
    },

    classLevelPermissions: {
        find: {},
        count: {},
        get: {},
        update: {},
        create: {},
        delete: {},
        protectedFields: {},
    },
}

var schemas = [IMessageReply, ....];

and then

router.get("/schema", async (req, res) => {
    for (let i = 0; i < schemas.length; i++) {
        const className = schemas[i].className;
        const fileds = schemas[i].fields;
        const keys = Object.keys(fileds);
        console.log("className: " + className);
        const schema = new Parse.Schema(className);

        var needCreate = true;
        try {
            const oldSchema = await schema.get();
            needCreate = false;
        } catch (e) {
            needCreate = true;
        }

        for (let j = 0; j < keys.length; j++) {
            const key = keys[j];
            const value = fileds[key];
            const type = value.type;
            const defaultValue = value.defaultValue;
            const targetClass = value.targetClass;
            if (type === "Number") {
                if (defaultValue) {
                    schema.addNumber(key, {defaultValue: defaultValue});
                } else {
                    schema.addNumber(key);
                }
            } else if (type === "Boolean") {
                if (defaultValue) {
                    schema.addBoolean(key, {defaultValue: defaultValue});
                } else {
                    schema.addBoolean(key);
                }
            } else if (type === "String") {
                if (defaultValue) {
                    schema.addString(key, {defaultValue: defaultValue});
                } else {
                    schema.addString(key);
                }
            } else if (type === "Date") {
                if (defaultValue) {
                    schema.addDate(key, {defaultValue: defaultValue});
                } else {
                    schema.addDate(key);
                }
            } else if (type === "Array") {
                if (defaultValue) {
                    schema.addArray(key, {defaultValue: defaultValue});
                } else {
                    schema.addArray(key);
                }
            } else if (type === "Pointer") {
                if (defaultValue) {
                    schema.addPointer(key, targetClass, {defaultValue: defaultValue});
                } else {
                    schema.addPointer(key, targetClass);
                }
            }
        }

        if (needCreate) {
            await schema.save();
            logInfo("create " + className + " success");
        } else {
            await schema.update();
            logInfo("update " + className + " success");
        }
    }

    logInfo("create schema success");
    res.send("Hello World");
})

if schema exist, call schema.update, or schema.save.

parse-community / parse-server