Is it possible to use sequelize operators (symbols)

[marc-at]

Sequelize now recommends using operators, which are javascript Symbols.

http://docs.sequelizejs.com/manual/tutorial/querying.html#operators-security

This causes a problem, because Symbols are ignored by JSON.stringify and Object.keys. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol

This cache implementation uses CircularJSON to generate the cache key, so it doesn't work when using Symbols as keys. https://www.npmjs.com/package/circular-json

Has anyone investigated a way to generate cache keys that includes Symbols?

If you need an example of this, I can provide one. To test without that, you can use the key function from Cacher:

  hash = crypto.createHash('sha1')
    .update(CircularJSON.stringify(this.options, jsonReplacer))
    .digest('hex');

the where object to pass would be something like

{
 [Op.or]: [
    {firstName: {[Op.like]: 'test'}},
    {lastName: {[Op.like]: 'test'}}
  ]
}

instead of

{
 $or: [
    {firstName: {$like: 'test'}},
    {lastName: {$like: 'test'}}
  ]
}

[rfink]

Hey there, and very interesting proposal. I've tried converting the app to generate the raw SQL to use as a unique cache key, but ran into many issues because there's currently no good way to generate the raw SQL from the JSON. I'm open to any ideas you have for solving this issue. Any thoughts?

[marc-at]

I've looked at

Reflect.ownKeys(where)

which will return the keys that are Symbols. I haven't worked out completely how this would work with nested queries or how to add/append these values to the generated key.

I am starting to dig through the sequelize code to see how it does it, but I am pretty new at sequelize, so I haven't found anything yet. My guess is that sequelize always looks for the known operators and doesn't have to look at Object.keys or something similar.

update: it looks like there is some code in sequelize/lib/utils.js:

//Collection of helper methods to make it easier to work with symbol operators

/**
 * getOperators
 * @param  {Object} obj
 * @return {Array<Symbol>} All operators properties of obj
 * @private
 */
function getOperators(obj) {
  return _.intersection(Object.getOwnPropertySymbols(obj || {}), operatorsArray);
}
exports.getOperators = getOperators;

/**
 * getComplexKeys
 * @param  {Object} obj
 * @return {Array<String|Symbol>} All keys including operators
 * @private
 */
function getComplexKeys(obj) {
  return getOperators(obj).concat(_.keys(obj));
}
exports.getComplexKeys = getComplexKeys;

[marc-at]

This might not be the best solution, but it passes my initial tests. This could also probably moved to an external module.

add the following imports:

const unsymbolize = require('./unsymbolize')
const _ = require('lodash')

change the key function to:

  hash = crypto.createHash('sha1')
    .update(CircularJSON.stringify(
      unsymbolize.unsymbolize(_.cloneDeep(this.options)),
      jsonReplacer))
    .digest('hex')

create a file named unsymbolize.js with this content:

'use strict'

/* eslint-disable no-use-before-define */
const unsymbolizeArray = (obj) => {
  if (obj.length) {
    for (let i = 0; i < obj.length; i++) {
      unsymbolize(obj[i])
    }
    return obj
  } else {
    return obj
  }
}

const unsymbolizeObject = (obj) => {
  const symbolKeys = Reflect.ownKeys(obj)
  if (symbolKeys.length) {
    for (let i = 0; i < symbolKeys.length; i++) {
      const key = symbolKeys[i]
      if (typeof key === 'symbol') {
        obj[key.toString()] = unsymbolize(obj[key])
      } else {
        obj[key] = unsymbolize(obj[key])
      }
    }
    return obj
  } else {
    return obj
  }
}

const unsymbolize = (obj) => {
  if (obj === null) {
    return obj
  }
  if (typeof obj === 'undefined') {
    return obj
  }
  if (obj.constructor === String) {
    return obj
  }
  if (obj.constructor === Array) {
    return unsymbolizeArray(obj)
  }
  if (obj.constructor === Object) {
    return unsymbolizeObject(obj)
  }
  return obj
}
module.exports.unsymbolize = unsymbolize

I should probably also move the lodash cloneDeep() call to the symbolize module.

Maybe this will help.