search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.17k stars 5.48k forks source link

GCE `update_pricing` fails if the cache dir does not exist. #42117

Closed fake-name closed 6 years ago

fake-name commented 7 years ago 
>>> self.cc.action(fun='update_pricing', provider='gce')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 492, in action
    return mapper.do_function(provider, fun, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1615, in do_function
    driver: self.clouds[fun](call='function')
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/clouds/gce.py", line 2629, in update_pricing
    with salt.utils.fopen(outfile, 'w') as fho:
  File "/usr/local/lib/python2.7/dist-packages/salt/utils/__init__.py", line 1373, in fopen
    fhandle = open(*args, **kwargs)  # pylint: disable=resource-leakage
IOError: [Errno 2] No such file or directory: '/var/cache/salt/cloud/gce-pricing.p'

client@rabbit:/usr/local/lib/python2.7/dist-packages/salt/cloud/clouds⟫ sudo ls /var/cache/salt/cloud
ls: cannot access /var/cache/salt/cloud: No such file or directory
client@rabbit:/usr/local/lib/python2.7/dist-packages/salt/cloud/clouds⟫ sudo ls /var/cache/salt/
master  minion

This is probably another symptom of my using of the API in a way not originally intended by it's writer, but it definitely is an issue, I think.

fake-name commented 7 years ago

https://github.com/fake-name/salt/commit/95d51a481ce2f91c90d4fb54fda3a1648711c475 fixes the issue, but I'm somewhat confused by the gce.py file. For example, os.path.* functions seem to be available, but os.path is never imported? There's a lot of really strange stuff going on with metaprogramming and map everywhere, and the resulting code is basically impenetrable.

Ch3LL commented 7 years ago

Seems this issue is resolved with your PR. Can you elaborate on what you are not understanding. Which parts of os.path are being used where? I can see os being imported at the top and references to os.path which seems correct. Thanks

fake-name commented 7 years ago

Huh, I had always assumed that to use os.path.* functions, you had to specifically import os.path, in addition to any import os import. I just tested it, and apparently I was incorrect in my assumption. Derp.

Never mind about that, I guess.

fake-name commented 7 years ago

More stuff:

Main.VpsHerder.MainThread - INFO - Generator call: <bound method VpsHerder.generate_gce_conf of <salt_runner.VpsHerder object at 0x6763fb691390>>
Main.MainThread - CRITICAL - Uncaught exception!
Main.MainThread - CRITICAL - Uncaught exception
Traceback (most recent call last):
  File "salt_scheduler.py", line 171, in <module>
    sched.ensure_active_workers()
  File "salt_scheduler.py", line 127, in ensure_active_workers
    self.create_vm(vm_name)
  File "salt_scheduler.py", line 55, in create_vm
    self.interface.make_client(vm_name)
  File "/home/client/AutoTriever/marshaller/salt_runner.py", line 485, in make_client
    provider, kwargs = self.generate_conf()
  File "/home/client/AutoTriever/marshaller/salt_runner.py", line 350, in generate_conf
    return selected_gen_call()
  File "/home/client/AutoTriever/marshaller/salt_runner.py", line 261, in generate_gce_conf
    image, size, places = self.get_gce_5_bux_meta()
  File "/home/client/AutoTriever/marshaller/salt_runner.py", line 228, in get_gce_5_bux_meta
    self.cc.action(fun='update_pricing', provider='gce')
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 492, in action
    return mapper.do_function(provider, fun, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1615, in do_function
    driver: self.clouds[fun](call='function')
  File "/usr/local/lib/python2.7/dist-packages/salt/cloud/clouds/gce.py", line 2624, in update_pricing
    price_json = http.query(url, decode=True, decode_type='json')
  File "/usr/local/lib/python2.7/dist-packages/salt/utils/http.py", line 604, in query
    ret['dict'] = json.loads(salt.utils.to_str(result_text))
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

It looks like there was a hiccup with https://cloudpricingcalculator.appspot.com/static/data/pricelist.json, which is what the module uses. Interestingly enough, I tried fetching it with wget and `curl, and I wound up with a compressed version of the file in both cases, so I think it's possible that the remote host is incorrectly setting it's headers somewhere, leading to the client failing to decompress the response.

Edit: Nope - 

4 client@rabbit:~⟫ wget -S https://cloudpricingcalculator.appspot.com/static/data/pricelist.json
--2017-07-08 09:37:19--  https://cloudpricingcalculator.appspot.com/static/data/pricelist.json
Resolving cloudpricingcalculator.appspot.com (cloudpricingcalculator.appspot.com)... 2a00:1450:400e:808::2014, 172.217.19.212
Connecting to cloudpricingcalculator.appspot.com (cloudpricingcalculator.appspot.com)|2a00:1450:400e:808::2014|:443... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: Sat, 08 Jul 2017 07:34:40 GMT
  Expires: Sat, 08 Jul 2017 07:44:40 GMT
  ETag: "Qm4Gjw"
  X-Cloud-Trace-Context: a5fbeeb55a0e18982b21ccce7ee51059
  Content-Type: application/json
  Content-Encoding: gzip
  Server: Google Frontend
  Content-Length: 7084
  Age: 39
  Cache-Control: public, max-age=600
  Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,36,35"
Length: 7084 (6.9K) [application/json]
Saving to: ‘pricelist.json.3’

100%[==============================================================================================================================>] 7,084       --.-K/s   in 0.002s

2017-07-08 09:37:19 (3.16 MB/s) - ‘pricelist.json.3’ saved [7084/7084]

It's properly setting the Content-Encoding: header. Huh.

fake-name commented 7 years ago

Yeah, digging through salt.utils.http, it looks like there's no provision for handling compressed content. I got trace logging running:

[TRACE   ] 'drill' could not be found in the following search path: '['/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/games']'
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: rabbit.wlnupdates.com
[DEBUG   ] Requesting URL https://cloudpricingcalculator.appspot.com/static/data/pricelist.json using GET method
[DEBUG   ] Response Status Code: 200
[TRACE   ] Response Headers: {'Content-Length': '7084', 'Content-Encoding': 'gzip', 'Age': '0', 'Expires': 'Sat, 08 Jul 2017 08:11:58 GMT', 'Server': 'Google Frontend', 'Connection': 'close', 'Etag': '"Qm4Gjw"', 'Cache-Control': 'public, max-age=600', 'X-Cloud-Trace-Context': 'a5fbeeb55a0e18982b21ccce7ee51059', 'Date': 'Sat, 08 Jul 2017 08:01:58 GMT', 'Alt-Svc': 'quic=":443"; ma=2592000; v="39,38,37,36,35"', 'Content-Type': 'application/json'}
[TRACE   ] Response Cookies: None
P;qզ2S3���n}���V�����ϫ���^�?��|����U���v��o}Z�W���>�n����Z�O�������j�s6�n��ek�Y�?{ح[_ַ׷�[�˻���r�����B����ˇ�]��úu����{��Zo:�������i����ǯ�^�=�n:>ޭ:�o�~Y?�쮷�_�w����ڶ�/��U�w���{��g��}U-�>���u�_�}Q�^�W������������_���X]ݩ�V������ovjn�W7W�q��/7+��a��V�|Vp�K����/�Ay�D����pII�Wdyᯪ���'�y��.���}9����������t���۔��x�ݦ}�Ծ^�1\-7[��ȷ*�ڭ������!���B�#��ýR␍UýýýF[ýýýý$ýHý▒46│4ýýý ýýý9ýý▒4ýý°4ý%ý6ýýýýHýý<ýý␋ýý0ý─H[ý┘&ý ýýý"$┌DýM$ýý▒$ZIý8␉$▒HýýXFýý␋$ý6ÑFý�8��ýýCýý└ ýýýýýýYýýýýYKýýýY+<┐E±ýýýý≠ýý⎻ý⎻ý0ý
ýýL0ýýý┤ýýF#ýý␋42JýD"ý␤├ý
ýýAý1ý&ý#ý␍"ýý6*ýý␍ý▒°642π°#ý±ý:!'ýýýýý,ýý─ý┌,ýý⎻N
ýýý1"ýGý≥6Bý7ýý@ýýý%"9ýýX2ý!␌³ýQ^ý)ýý±#:@ýýýDýýýýýýOýý┌␍G0ýL7ý¨ýýýM=ý\ý
                                                                    ý≤│5°␤ ý├ý8ýý9RR≥▒:ýýKý$ýCLýU ýýýý@Jýý2!+ý
ýCý@*Jýý0ý·J%8≠␉ýHýýýýý◆Í
Sýý◆ý┌ý◆ýýFSýýýTýJ»ýý1ý?◆ý+ýý␍Z@ýý:XXý]ýý ␌ýýF,ýý©┤@ýýýý]ýàNýýN'8"ý@ýý┌RýýI─ýýýýýý9ý≠ý %ýý^1±"ýFýý'ý

                                                                             ýýýUG▮ý8ýý'Tý V␉3⎻ý┼ýýPýýý]ýý'ý>πýýýý$#ýýý≥ý¦ýýýýýýýI┤ýýý␋ý Zý␊ý≠ýýX;-ýýýZ␍ýý9ýýý@·ýGý^ýýýýýUJý␉ý"≥F:├,ýXýSý-Œýýý┐ý┬TMý├ý%ýý£ýý)ýý'ýπýýý2⎺▒Gý#@ýUN(2␋ýýýýýE        $ý*ýýýýýØýýýýPýLýáýýý┼ýýS(ýOýýýýýý3ýýýýýý⎻<-ýý▮⎽ýý⎽ý┬ý£L┼ýýýýýýýý&±├ýý<ý'◆ý├ýP=ý≤ý┼µ#6⎻ý└ýýýýýýýÿý┴ýB⎺!ý>ýýπ9ýOýýý9┴ý▮Gý┤ýWýý▒Ný#▮ýJýýý1ýýý¸ý5Lý:ýý┴[%$ý┐/⎽ý)Xý␊ýýýýý\ýý!ýýý⎼ýý+CSý┌FýM&␤ýý9ý8.ý±│ý◆ýBý└ýý+ýQ7ýý!ý3D⎼ýýýýF⎽3ý┐+3SýIFýM&°ý≤ýý;Iý2┘ýý     Eýýý␤ýý£
ýýG┼ýýýý┴⎼┬;ýý!6ý'.ý"›ý#LýùKýý%·K[ý)6ýπýNý>Eý[≤─│┼ý⎼ý P▮ýý⎼ý1ýý␤]IýýýýLý
(6␍W◆Rý0ýý'ýýCý⎽·ýýM£ýýýýπ⎻ýRýý⎼ýýý@ýýý┤ýýý;ý␌1;TKý·ýýýZýF┼ý⎽Mýý'ýúý4?ý,?ý$ýHRýýýý!ý:VZýZ≠ý␌.:ýý┼ý⎺▒┐]ýJHý'≥ýýýýSý<£ýýý3ýëýý'ýGýýý=P$XE└ý°ýýýýýýý└ýýýýý™ý88ýýý,M▮ýFýýýýýý*·
   #ýýýQýýÛýHý
ýRýýýý␉├ý°Qýý&ý(.␋ý≤ý≤ýý┌Dý≠
ýý/ýK␤ýý´Xý°┐ý7␍▒@
ý⎼ýR┌ýý$└ýý├ý@$ýýý/ÿýýJý┼M±ý¾ýý┐ýý.ý=≠÷≤XýýýOý?ýJý*ýýS┴≥@ý]ýVýýý´V⎺ý└›ýýý┘┼·≥ýýý;ý␍ý␋┤Gý,ý≥┴ý⎻⎻⎺ýý°ýýý┘ý+ýý┼<ý≠┌ýýýý└⎺ý▮ýý¶─ýýýý]πý]ýý,ý7│ý,ý␤��!G�h���=1���M�S�U�j�e}K�\�)ֺy����o�l>�k���f��۟��f��d�����<�5���a��c��� �4��9��1���7�^(I�7C���KBwQIr�}��UO��L�l��;�\��V�����o�|6nj���F�d�j�ބ;�Nýý┐ýýý·.;␊ýýýý┼ýý£“ýWý>ýýýýýE1ý≤ý;ý°ý┴ýRý␌V‰ý&ý±┴␊␋ý␉ýπF[ýý3ýý▒┴ýýýýOýý5ýÿ⎽ý┬.ýý/┴ýýýýýýýý2/≥ýýýI>]ýý┴/@±ýDý⎼[0ý°ýýGý^1ý≤ýýG│⎻ý▒ý6ýCý8ýý+ý┌ýýý␤ýýýýýýQ%ýý1ýý ýýHGG┐9ýýýýýA=Hý6@ý!ýý=QD� c�$�
�(����W����O��/�]̦9)�Ǿ�_���H�L��>�vo�����(Ϧ����sW��!bO�c�u�&_��/0���������Ȑ|9��ɦ����j��W�W��"�:�f?��Ie�(�����fk�ޛ���N�4��hýýý▮;≠5ý⎺ýýýýý,ýýýN␍␋ýýLýý]ýEýý▒ý#␋ýýS/ý>Hýý⎺°’ý,┐VýS¢ýý ýý≥=┴ýý␍Oýý±­ýýøK)¥ýýý␉≠7ýRýýý┘ý3ýýYý◆!┐▒┐ýý<ý*Mýý⎼ýãLýýýýý]ý◆ýý│°^
ý,ý◆ýSπBýýäý8°┌ý             ýýKQýôK\ý]ýýN�Rw<��y{1S�7ٓv2n�/�_�&�,�qԆ↓ýFýýýýýýý4ýýGý┼ýM&ýý]ý£=ýýýBý▒ýZýýπý.Mýýý[ý⎺⎺ýEý+  YýS:ý/ý␌5);ýý␊
ýý8ý°ýýýBýý$┤ýý≠ýýýý2≥ýý␉ýýýýýýýQý┐⎽ý[7ýn┴>▒ý°≤ýýýýýýý6ý· ý≤6πýý└ý/·ýU�X6���f�A��?�/2`N@�ϕ�{��2�q�='�_ԟ�Ŗ��7��\�B���wL?e��0�ϡ�4��i��~R��e��ߕ=��͗�O��ʣ 9��^>����S�w���
                                                                                                                                                                    ��Oߵ"�����w�u�T�]���o�?��7��k�Z�<�?ڻ6�ۍ8�|��j��V�Ӏ}�򱅿�B}o���o\���U�������l��)F�3���3�!hE;A;�fz��?��i�뗍j��DŽ        �;Nk�|1x]@���MiHMz6W��N�-���t���0���E�� ��w��y�b"|9�i�g���ew�@x�54Ϟ�/���j("�,ǎ�XO�\���n"�s�$C�7x������x*4 j��3���?�]�Y�dG�]n��Ѯ��Oܶr��� o��=������^�bΐ�z��J�dqޞ�������sR���N��b}�QSe��
���
8���:�d����M��/d2�9���\-��ý³ýý@H7ý␍£ý
CGýýýý0ýýý─HýýM9ýýýý°Sýý"ýý7ý┌ý8ýýý*Eý⎺ý¿ ý├<└ýý␉␉ý,,GýýG*.=┌▮ýý)ýNýýýýýýýýý⎻ýý(‡)±┘␍ý:Yýý┐ýZ\ýUF$ý.ýU9Pý␊™HýýýR&O(┌ý
U±ýýýý·ý␉ýEÈ⎼ý␉ýýýIý⎻(3
                       ýý-&ýý◆ýýý$ý⎽ýýý-59ý9ý·ý␌┘ý┘,H]ýE
                                                        5ýAý˜ý&π/PJýý6ý◆,ý4ý┌ýý˜≥≤=ý#
⎼ýý⎽ýýý?└ý=ýýýýC£ýýPýý                                                               ␍ý└ýý5ý⎻ýýý┐;0ýý┌Xýýýýý,ýýýý?ýýý^Sýý┐
Rý#TGý⎼ýOýýYýý▒@ý$ýý␌VýL┼)␍@│ýKL1␍<␍─9M­ýMSýýý≥ýýýý⎼▮ýýA┴ý�'�a�44��L���y&��UW���S6_���@�v2�w��2�/��?dX��z�u��R��7�����gӟ�����e��2-2q�fU�'�͛���G8:$�T�џ�d�\���3�hu_����|��G����+����c8$H^N�EI���뒳�|>-
D�|֝�'���W��_t,��Z7I�۝��� -J���Q�dJ��L�
���>������h��}�n�� Ǔ�7�.�8:ʈ���/�J����A6��f���_y#�(Q�9F�� � F��G��{�[�^��?0�賹�9.'�&�V=�]-��l�
��>�~�p�"I���� �a8��F�Q��u��&Rn��q�&��Z�'A��ȓH�$`�gw<����_q�����pt�.�pt|�љ�l�u
>�h�:P���s5-;�>ݠV�p1����[C245�ȧwPՁG?��>����M���VO=��1u)5�Ԁ
�k4��t��x���w:cc��hx�i���r&���f�}5��x1ŷtA��U���rs�mz�,���گ�w��|:#P��B�;}7���\5���7Y�$��Z�4R>(�� a
ý␉ý)ý⎺-ýýý°ýýý⎺ýýPýýE3␍L$┌Rý Tý°HXýýý␍⎽⎺¦ýý┌2ý┬ýýý≠ý�N�9�~0�q(��i�;�R�o��kyz���9�ͪ;�����j�,��Z�Z�oýýýLý␤ý▮ýýýýý≥ýý:ý7P≥ý-@ýýý
                                                                                                                            ýýýý["<ýý┌&.!t:�G   ���!t0lQ��F���~�§6�Պ����>6��QV��L��ݘrFt�Rg:���(5�lΗ��ȵ<���O2lB��        �a��
                                                    *~��w�Y}l�����Kh��!g��R�J��x�Ң��    2e�hW�3�i����Q'�`�t!��䮓���V�B�\_�0�LCI��ď�b�h�{Rʞ��ҏ�v|�*Eΐ<j�z3��6��vx3�CW��� �:s�~���P�[�=�t)y�u��ɛ���X�R���/���l7����
[ERROR   ] An un-handled exception was caught by salt's global exception handler: