A MongoDB query string parser to be used in applications developed with NestJS.
As the name of the library suggests, it was built to work together with the NestJS framework.
Use the follow command:
npm i --save nest-mongo-query-parser
There are two ways to use the parsers available in this library: as a ParamDecorator or as a MethodDecorator.
If you want to use it as a ParamDecorator, just add the tag referring to the Parser to be used as a method parameter. Example:
import {Get} from '@nestjs/common';
import {Controller} from '@nestjs/common';
import {ResourceService} from './resource.service';
import {MongoQuery, MongoQueryModel} from 'nest-mongo-query-parser';
@Controller('resources')
export class ResourceController {
constructor(private readonly _service: ResourceService) {
}
@Get()
public find(@MongoQuery() query: MongoQueryModel) {
return this._service.find(query);
}
}
It can also be used as a MethodDecorator. Just use the tag referring to the Parser to be used as the method decorator. Example:
import {Injectable} from '@nestjs/common';
import {MongoQueryParser, MongoQueryModel} from 'nest-mongo-query-parser';
@Injectable()
export class ResourceService {
@MongoQueryParser()
public find(query: MongoQueryModel) {
return [];
}
}
NOTE: When using the library as a MethodDecorator, you can receive other arguments in the method in question, but the query has to be passed as the first argument of the function, so that the treatment is done properly.
{
"limit": 100,
"skip": 0,
"select": {},
"sort": {},
"populate": [],
"filter": {}
}
{
"limit": 10,
"skip": 10,
"select": {
"_id": 1,
"name": 1,
"age": 1
},
"sort": {
"created_at": -1
},
"populate": [],
"filter": {
"age": {
"$gt": 30
}
}
}
The paging feature is very useful for clients who will consume your API. It is through this feature that applications can define the data limit in a query, as well as define which page to be displayed. Each time a page of an application is selected, it means that some resources have been displaced (data offset or skip data).
There is a mathematical rule that relates page number to resource offset. Basically:
offset = (page - 1) * limit, where page > 0.
This means that for a limit of 10 elements per page:
And so on.
With this library, it is possible to use pagination with the page
parameter, or using the skip
manually. By default,
the limit
value is 100
and skip
value is 0
.
Example:
{
"limit": 10,
"skip": 20
}
{
"limit": 10,
"skip": 20
}
To work with ordering, you need to specify one or more sorting parameters, and whether you want the sorting to be ascending or descending. For ascending ordering, just put the name of the ordering parameter. For descending ordering, you need to put a "-" symbol before the name of the ordering parameter. Example:
{
"sort": {
"created_at": 1
}
}
{
"sort": {
"created_at": -1
}
}
{
"sort": {
"age": -1,
"name": 1
}
}
In multiple-parameter ordering, the first ordering parameter has higher priority than the second, and so on. In the
example above, the ordering will be given primarily by the age
parameter, in descending order. If there are two or
more objects with the same value in age
, then those objects will be sorted by name
in ascending order.
With this library, you can choose which parameters should be returned by the API. However, Mongo has a peculiarity: you can also specify which parameters you don't want to be returned. The logic is similar to ordering: to specify which parameters are to be returned, simply enter the parameter name; and to specify which parameters should not be returned, just place a "-" symbol before the parameter.
Example:
{
"select": {
"_id": 1,
"name": 1,
"age": 1
}
}
{
"select": {
"_id": 0,
"created_at": 0,
"updated_at": 0
}
}
It is interesting to use one or the other in your queries, as one is complementary to the other. If you want almost all parameters except a few, use the option to ignore parameters. If you want some parameters, and ignore the others, use the option to select the ones you want.
Now let's go to the most complex part of the library: the filters. There are several ways to apply filters in this library, so I'm going to break this topic down into subtopics for every possible filter approach.
Simple filters are equality filters. Basically it's set key=value. All filter parameters are defined as string, so there are some validations that are done on these values.
If the value is a string number, it is transformed into a number, either integer or float/double (up to 16 decimal places);
If the value is in yyyy-MM-dd format or yyyy-MM-ddThh:mm:ss.sZ format, it is transformed into a Date object;
If the value is 'true' or 'false', it is transformed into a boolean value, according to your value;
Otherwise, the value is considered as a string.
Example:
{
"filter": {
"name": "John Doe",
"age": 31,
"birth_date": 1990-01-01T00:00:00.000Z
}
}
You can specify multilevel filters. This means that, if you have an object that has a field that is another object, you can perform a search with filters through the parameters of the internal object. Example:
{
"_id": "613532a350857c1c8d1d10d9",
"name": "Filippo Nyles",
"age": 28,
"current_job": {
"title": "Budget/Accounting Analyst III",
"salary": 4776.8
}
}
{
"filter": {
"current_job.title": "Budget/Accounting Analyst III"
}
}
Partial filters are a way to search a string type value for a part of the value. There are three ways to use partial
filters. Making an analogy with javascript, it would be like using the startsWith
, endsWith
and includes
methods,
where:
Example:
{
"filter": {
"name": {
"$regex": "^Lu",
"$options": "i"
},
"email": {
"$regex": "gmail.com$",
"$options": "i"
},
"job": {
"$regex": "Developer",
"$options": "i"
}
}
}
Comparison operators are specific filtering options to check whether a parameter has a value. It is possible to check not only equality, but other mathematical operators, such as: ">", ">=", "<", "<=", "!=". In addition, you can use comparison operators to check whether an element is in an array.
According to the mongodb documentation, the available comparison operators are:
To use these operators, just pass the comparator tag without the "$" symbol. Example:
{
"filter": {
"age": {
"$gt": 30
}
}
}
I won't put an example with all operators here, but you can test arithmetic comparison operators on parameters with
values of type string or number, or test the operators of $in
and $nin
on parameters of type array.
Element filters are filters used to deal with parameters that make up the entity's schema. There are two types of element filter possibilities:
$exists: returns elements that have or do not have a specific field
$type: returns elements whose field has a specific type.
Example:
{
"filter": {
"created_at": {
"$exists": true
},
"updated_at": {
"$exists": false
},
"jobs": {
"$type": "array"
}
}
}
The $exists filter only works with true
or false
values. If a different value is entered, the filter will be
ignored.
The same goes for the $type filter, which only works with valid type values defined in the mongodb documentation ( except deprecated ones):
{
"validTypes": [
"double",
"string",
"object",
"array",
"binData",
"objectId",
"bool",
"date",
"null",
"regex",
"javascript",
"int",
"timestamp",
"long",
"decimal",
"minKey",
"maxKey"
]
}
Finally, it is possible to use filters with AND | OR operator. The usage logic follows the arithmetic rule.
To use the AND operator, you must pass the same value twice in a query. Example:
{
"filter": {
"$and": [
{
"age": {
"$gt": 30
}
},
{
"age": {
"$lt": 50
}
}
]
}
}
To use the OR operator, you must enter the values separated by a comma. Example:
{
"filter": {
"$or": [
{
"age": 30
},
{
"age": 50
}
]
}
}
If any collection uses references to other objects, in some operations it is interesting to return this information
populated in the object in a single request. For this, the library supports the populate
feature.
There are three ways to add the populate
parameter to the query string:
{
"populate": {
"path": "jobs"
}
}
{
"populate": {
"path": "jobs",
"select": {
"title": 1,
"salary": 1
}
}
}
{
"populate": {
"path": "job",
"select": {
"title": 1,
"salary": 1
},
"match": {
"salary": {
"$gt": 3000
}
}
}
}
{
"populate": [
{
"path": "jobs"
},
{
"path": "currentJob"
}
]
}
There are some rules to consider in populate. The populate must be specified as follows:
populate=field;select;filter
. Soon:
select
parameter must be informed as all
.
Example: populate=jobs;all;salary=gt:3000
Optionally you can override default search filter behavior by passing some specific options with the MongoQuery()
decorator. This will be used to search in all specified fields of the document.
import {Get} from '@nestjs/common';
import {Controller} from '@nestjs/common';
import {ResourceService} from './resource.service';
import {MongoQuery, MongoQueryModel} from 'nest-mongo-query-parser';
@Controller('resources')
export class ResourceController {
constructor(private readonly _service: ResourceService) {
}
@Get()
public find(@MongoQuery({
search: {key: 'q', paths: ['name', 'description']},
}) query: MongoQueryModel) {
return this._service.find(query);
}
}
{
"limit": 30,
"skip": 0,
"select": {},
"sort": {},
"populate": [],
"filter": {
"$or": [
{
"name": {
"$regex": "John",
"$options": "i"
}
},
{
"description": {
"$regex": "John",
"$options": "i"
}
}
]
}
}
Sometimes, we need to add some parameters to the ordering, selection, filters and even population objects in the query in the code, for some cases that we need to inform some query params that weren't informed by the client. For that, this new feature has been added.
To use it, it's simple: just call the method corresponding to the resource you want to add. The methods signatures are:
interface MongoQueryModel {
addSort(sort: QueryObjectModel): void
addSelect(select: QueryObjectModel): void
addFilter(filter: QueryObjectModel): void
addPopulate(populate: QueryObjectModel | QueryObjectModel[]): void
}
An example of use: let's assume that we have the following MongoQueryModel object, and that it is assigned to a variable called "object".
const object: MongoQueryModel = {
filter: {age: {$gte: 30}},
sort: {created_at: -1},
select: {name: 1, age: 1},
populate: {path: 'job'}
}
addFilter()
method. Then, we have:object.addFilter({ "name": { "$regex": "Smith", "$options": "i" }})
addSort()
method. Then, we have:object.addSort({ age: 1 })
addSelect()
method. Then, we have:object.addSelect({ gender: 1})
addPopulate()
method. Then, we have:object.addPopulate({ path: address })
So the final object variable would become:
const model: MongoQueryModel = {
filter: {age: {$gte: 30}, name: {$regex: 'Smith', $options: "i"}},
sort: {created_at: -1, age: 1},
select: {name: 1, age: 1, gender: 1},
populate: [{path: 'job'}, {path: 'address'}],
}
Rules:
Priority will be given to fields that are added using the methods of the MongoQueryModel object. Therefore, if you
add a new parameter whose key already exists in the filter, it will be replaced by the new value to be added.
So if you have the filter { age: {$gte: 30 }, name: { $regex: 'Smith', $options: 'i' } }
and add the
filter {age: 10}
, the final result of the filter will be { age: 10, name: { $regex: 'Smith', $options: 'i' } }
This rule is valid for all query parameters with type ObjectQueryModel.
For the populate property, we have a few more rules:
addPopulate()
method is an array, both arrays will be merged. So if
you
have populate param as [{ path: 'job' }, { path: 'address'}]
and call the method with the
parameter [{ path: 'school_address' }, { path: 'job_address' }]
, the final result will
be [{ path: 'job' }, { path: 'address'},{ path: 'school_address' }, { path: 'job_address' }}]
.addPopulate()
method is an object, it will be transformed into an array, and the
current value will be added to the new values. So if you have the populate param
as [{ path: job }, { path: address}]
and call the method
with the
parameter [{ path: job_address }]
, the final result will
be [{ path: job },{ path: address }, { path : job_address }}]
For cases where the current populate is an object:
If the populate param to be informed in the addPopulate()
method is an array, the current value will be
added to the
new
values. So if you have the populate { path: job }
and call the method with the
parameter [{ path: school_address }, { path: job_address }]
, the final result will
be [{ path: school_address }, { path : job_address }}, { path: job }]
If the populate informed in the addPopulate()
method is an object, the main rule will be checked and, if
they
differ,
an array will be formed with both current and new populate values. So if you have the populate { path: job }
and call the method with the parameter { path: 'address'}
, the
final result
will be [{ path: job }, { path: address}}]
. And if you have the populate { path: job }
and call the method
with the
parameter { path: 'job', select: 'all'}
, the final result
will
be { path: 'job', select: 'all'}
limit
, skip
and page
only;sort
only;select
only;populate
only;limit
, skip
, page
, sort
, select
and populate
will be considered a filter;/[^A-z0-9_.]/g
;/[^\w\s@.-:\u0600-\u06FF]/g
;
\u0600-\u06FF
contains arabic charactersThis library is generic. This means that it handles the query based on the query object itself. Therefore, it is not possible to control events such as filter parameters with types incompatible with the types defined in the base. Use proper queries for your API, to prevent implementation errors from being thrown into your app.
Check out how the configuration of the library in an API works in practice in this project.
Distributed under the Apache License 2.0. See LICENSE
for more information.