fog / fog-core

fog's core, shared behaviors without API and provider specifics
MIT License
45 stars 95 forks source link

Cache load #284

Closed jsirex closed 1 year ago

jsirex commented 1 year ago

Two commits, no need to squash.

Fixes cache load with aliases/anchors. Uses temporary directory for cache tests.

geemus commented 1 year ago

I don't think we write anything with aliases or anchors do we? Would this be for a case where someone has modified the cache themselves or am I missing something?

jsirex commented 1 year ago

There is long description in commit body:

This patch enables anchors/aliases when loading YAML.

It helps with the situation when you receive a big object (like
json response) and want to split it into attributes where some
attributes actually can point on the same object in memory. That forces
Pshyc.dump to produce anchors in the resulting YAML file.

Potentially, there are other cases when anchors can appear in YAML.
But by default alias parser is disabled. That means you cannot load
your cache.

YAML behavior is different after ruby >=3.1.0.

If you have attributes hash with multiple values which are actually pointers on the same object, you will get reference. As result it will be dumped as alias.

# /tmp/test.rb
require 'yaml'

puts RUBY_VERSION

fake_response = {'one' => {foo: 5}, 'two' => {bar: 6}}
model_attributes = {}
model_attributes[:first_attribute] = fake_response['one']
model_attributes[:second_attribute] = fake_response['two']
model_attributes[:full] = fake_response

result = YAML.dump(model_attributes)
puts result

hash = YAML.load(result)
puts hash.inspect
╭─sirex at sirex-notebook in ~/proj/jsirex/fog-core on attributes-methods✔ using ‹ruby-2.7.0@fog-core› 23-05-26 - 1:08:11
╰─± ruby /tmp/test.rb
2.7.0
---
:first_attribute: &1
  :foo: 5
:second_attribute: &2
  :bar: 6
:full:
  one: *1
  two: *2
{:first_attribute=>{:foo=>5}, :second_attribute=>{:bar=>6}, :full=>{"one"=>{:foo=>5}, "two"=>{:bar=>6}}}
╭─sirex at sirex-notebook in ~/proj/jsirex/fog-core on attributes-methods✔ using ‹ruby-2.7.0@fog-core› 23-05-26 - 1:08:14
╰─± rvm use 3.2.1@fog-core
Using /home/sirex/.rvm/gems/ruby-3.2.1 with gemset fog-core
╭─sirex at sirex-notebook in ~/proj/jsirex/fog-core on attributes-methods✔ using ‹ruby-3.2.1@fog-core› 23-05-26 - 1:08:24
╰─± ruby /tmp/test.rb     
3.2.1
---
:first_attribute: &1
  :foo: 5
:second_attribute: &2
  :bar: 6
:full:
  one: *1
  two: *2
/home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:432:in `visit_Psych_Nodes_Alias': Alias parsing was not enabled. To enable it, pass `aliases: true` to `Psych::load` or `Psych::safe_load`. (Psych::AliasesNotEnabled)
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:30:in `visit'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:6:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:35:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:347:in `block in revive_hash'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `each'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `each_slice'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `revive_hash'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:169:in `visit_Psych_Nodes_Mapping'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:30:in `visit'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:6:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:35:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:347:in `block in revive_hash'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `each'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `each_slice'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:345:in `revive_hash'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:169:in `visit_Psych_Nodes_Mapping'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:30:in `visit'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:6:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:35:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:320:in `visit_Psych_Nodes_Document'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:30:in `visit'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/visitor.rb:6:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych/visitors/to_ruby.rb:35:in `accept'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych.rb:334:in `safe_load'
        from /home/sirex/.rvm/rubies/ruby-3.2.1/lib/ruby/3.2.0/psych.rb:369:in `load'
        from /tmp/test.rb:14:in `<main>'
geemus commented 1 year ago

Thanks for the explanation on the changes in newer rubies, I didn't realize that behavior had changed.