search

meelgroup / barbarik

The first efficient procedure to test whether a sampler is uniform
MIT License
13 stars 1 forks source link

Problems to run barbarik #6

Closed rheradio closed 4 years ago

rheradio commented 4 years ago

Hi,

I've just downloaded barbarik. Following your instructions, I've tried to run it:

./barbarik.py --seed 1 --sampler SAMPLER_TYPE blasted_case110.cnf

Then, the following error message appeared:

Experiment: 0     of     1
constantFactor:38   boundFactor: 10.219408528680095   logBoundFactor: 3.3532397943494647  
tj: 1789   totalLoops: 2     beta: 0.8181818181818182 epsilon: 0.6 
numSolutions: 389   loThresh:56     hiThresh: 332
Traceback (most recent call last):
  File "./barbarik.py", line 734, in <module>
    barbarik()
  File "./barbarik.py", line 710, in barbarik
    ok, breakExperiment = exp.one_experiment(experiment, j, i, numExperiments, tj)
  File "./barbarik.py", line 606, in one_experiment
    **chainFormulaSetup = chainFormulaSetup(sampleSol, unifSol, self.numSolutions)**
**UnboundLocalError: local variable 'chainFormulaSetup' referenced before assignment**

The python version installed in my Ubuntu computer is 3.7.3

Thank you so much.

Best regards, Ruben

msoos commented 4 years ago

Hi!

Wow, good catch. Let us try to fix this :) We'll get back to you soon,

Mate

PS: please use backtick to quote entire paragraphs. Just check how I edited yours :)

rheradio commented 4 years ago

Hi,

Thank you so much for your quick response.

I've just tried to run barbarik on another computer with Python 2.7.17. and I get a different error:

./barbarik.py --seed 1 --sampler 1 blasted_case110.cnf
Experiment: 0     of     1
constantFactor:38.0 boundFactor: 10.2194085287        logBoundFactor: 3.35323979435       
tj: 1789.0 totalLoops: 2     beta: 0.818181818182 epsilon: 0.6       
numSolutions: 389   loThresh:56     hiThresh: 332   
Solution list: ['-10 13 -15 -16 -25 -28 -39 41 43 45 5 -53 -6 -69 78 9 93 0']
Executing : ./spur -q -s 1 -out /tmp/blasted_case110.out -cnf blasted_case110.cnf
Traceback (most recent call last):
  File "./barbarik.py", line 747, in <module>
    barbarik()
  File "./barbarik.py", line 723, in barbarik
    ok, breakExperiment = exp.one_experiment(experiment, j, i, numExperiments, tj)
  File "./barbarik.py", line 619, in one_experiment
    chainFormulaConf = chainFormulaSetup(sampleSol, unifSol, self.numSolutions)
  File "./barbarik.py", line 322, in chainFormulaSetup
    sampleSol = sampleSol.strip()
AttributeError: 'list' object has no attribute 'strip'

Thank you so much!

Best, Ruben

msoos commented 4 years ago

Hi,

Ah, this has been a mess. Thankfully, Priyanka Golia has stepped up and fixed this in another branch. I am porting this here, and it should resolve all your issues. Please pull and try again :) Let us know how it went and close the issue if appropriate :) Thanks for using Barbarik,

Mate

rheradio commented 4 years ago

Hi,

Thank you so much for your effort. I've just tested barbarik's new version.

It works with all samplers, except with QuickSampler that produces the following error:

Experiment: 0 of 1 constantFactor:38 boundFactor: 10.219408528680095 logBoundFactor: 3.3532397943494647 tj: 1789 totalLoops: 2 beta: 0.8181818181818182 epsilon: 0.6 numSolutions: 389 loThresh:56 hiThresh: 332 Using sampler: QuickSampler Traceback (most recent call last): File "./barbarik.py", line 825, in ok, breakExperiment = exp.one_experiment(experiment, j, i, numExperiments, tj) File "./barbarik.py", line 712, in one_experiment self.inputFile, 1, self.samplerType, self.indVarList, newSeed) File "./barbarik.py", line 126, in getSolutionFromSampler sols = SolutionRetriver.getSolutionFromQuickSampler(*topass_withseed) File "./barbarik.py", line 231, in getSolutionFromQuickSampler with open(inputFile+'.samples', 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: 'blasted_case110.cnf.samples'

Thank you.

Best, Ruben

El dom., 3 may. 2020 a las 17:20, Mate Soos (notifications@github.com<mailto:notifications@github.com>) escribió:

Hi,

Ah, this has been a mess. Thankfully, Priyanka Golia has stepped up and fixed this in another branch. I am porting this here, and it should resolve all your issues. Please pull and try again :) Let us know how it went and close the issue if appropriate :) Thanks for using Barbarik,

Mate

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/meelgroup/barbarik/issues/6#issuecomment-623126078, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOLG2Y25SAIPZO7FDZBB5LRPWDUFANCNFSM4LRASJBQ.

--

Prof. Rubén Heradio Gil Universidad Nacional de Educacion a Distancia

Dpto. Ingeniería de Software y Sistemas Informáticos E.T.S.I. Informática, UNED C/ Juan del Rosal, 16 Ciudad Universitaria 28040 Madrid Spain

Phone: (+34) 91 398-8242 e-mail: rheradio@issi.uned.esmailto:rheradio@issi.uned.es Fax : (+34) 91 398-9382 internet: http://www.issi.uned.es/ Researchgate: https://www.researchgate.net/profile/Ruben_Heradio

AVISO LEGAL. Este mensaje puede contener información reservada y confidencial. Si usted no es el destinatario no está autorizado a copiar, reproducir o distribuir este mensaje ni su contenido. Si ha recibido este mensaje por error, le rogamos que lo notifique al remitente. Le informamos de que sus datos personales, que puedan constar en este mensaje, serán tratados en calidad de responsable de tratamiento por la UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA (UNED) c/ Bravo Murillo, 38, 28015-MADRID-, con la finalidad de mantener el contacto con usted. La base jurídica que legitima este tratamiento, será su consentimiento, el interés legítimo o la necesidad para gestionar una relación contractual o similar. En cualquier momento podrá ejercer sus derechos de acceso, rectificación, supresión, oposición, limitación al tratamiento o portabilidad de los datos, ante la UNED, Departamento de Política Jurídica de Seguridad de la Informaciónhttps://www.uned.es/dpj, o a través de la Sede electrónicahttps://sede.uned.es/ de la Universidad. Para más información visite nuestra Política de Privacidadhttps://descargas.uned.es/publico/pdf/Politica_privacidad_UNED.pdf.

priyanka-golia commented 4 years ago

Hi Ruben,

Can you please remove > /dev/null 2>&1 from quicksampler and z3 command, and share the output with us? Basically you need to make following changes in barbarik.py: line 221: cmd = "./samplers/quicksampler -n "+str(numSolutions*5)+' '+str(inputFile)+' > /dev/null 2>&1' to
cmd = "./samplers/quicksampler -n "+str(numSolutions*5)+' '+str(inputFile) . And, line 226: cmd = "./samplers/z3 "+str(inputFile)+' > /dev/null 2>&1' to
cmd = "./samplers/z3 "+str(inputFile) Then execute : ./barbarik.py --seed 1 --sampler 2 blasted_case110.cnf

This will help us to identify the exact issue.

Thanks ! Priyanka

rheradio commented 4 years ago

Hi Priyanka,

after making the changes, the following error is shown:

Experiment: 0 of 1 constantFactor:38 boundFactor: 10.219408528680095 logBoundFactor: 3.3532397943494647 tj: 1789 totalLoops: 2 beta: 0.8181818181818182 epsilon: 0.6 numSolutions: 389 loThresh:56 hiThresh: 332 Using sampler: QuickSampler 11100000011010111 STARTING Samples 2 Execution time 0.02155 unsat Samples 2 Execution time 0.0225686 Samples 4 Execution time 0.0304489 unsat Samples 4 Execution time 0.0313908 Samples 8 Execution time 0.0391858 Stopping: samples Samples 8 Execution time 0.0392059 Solver time: 0.0342901 Epochs 0, Flips 3, Unsat 2, Calls 6 Step inf s Probability 1.000000 Mutations 0 0 0 0 1 0 0 0 2 0 0 0 3 0 0 0 4 0 0 0 5 0 0 0 6 0 0 0 All 0.000000 / 0 = -nan Valid Invalid Total 0.000014 s Solver 0.000000 s Checked 0 Calls 0 Traceback (most recent call last): File "./barbarik.py", line 825, in ok, breakExperiment = exp.one_experiment(experiment, j, i, numExperiments, tj) File "./barbarik.py", line 712, in one_experiment self.inputFile, 1, self.samplerType, self.indVarList, newSeed) File "./barbarik.py", line 126, in getSolutionFromSampler sols = SolutionRetriver.getSolutionFromQuickSampler(*topass_withseed) File "./barbarik.py", line 231, in getSolutionFromQuickSampler with open(inputFile+'.samples', 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: 'blasted_case110.cnf.samples'

Thank you!

Best, Ruben

El lun., 4 may. 2020 a las 15:06, Priyanka golia (notifications@github.com<mailto:notifications@github.com>) escribió:

Hi Ruben,

Can you please remove > /dev/null 2>&1 from quicksampler and z3 command, and share the output with us? Basically you need to make following changes in barbarik.py: line 221: cmd = "./samplers/quicksampler -n "+str(numSolutions5)+' '+str(inputFile)+' > /dev/null 2>&1' to cmd = "./samplers/quicksampler -n "+str(numSolutions5)+' '+str(inputFile) . And, line 226: cmd = "./samplers/z3 "+str(inputFile)+' > /dev/null 2>&1' to cmd = "./samplers/z3 "+str(inputFile) Then execute : ./barbarik.py --seed 1 --sampler 2 blasted_case110.cnf

This will help us to identify the exact issue.

Thanks ! Priyanka

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/meelgroup/barbarik/issues/6#issuecomment-623451902, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOLG22GLBR4L23XSTYYH33RP24VDANCNFSM4LRASJBQ.

--

Prof. Rubén Heradio Gil Universidad Nacional de Educacion a Distancia

Dpto. Ingeniería de Software y Sistemas Informáticos E.T.S.I. Informática, UNED C/ Juan del Rosal, 16 Ciudad Universitaria 28040 Madrid Spain

Phone: (+34) 91 398-8242 e-mail: rheradio@issi.uned.esmailto:rheradio@issi.uned.es Fax : (+34) 91 398-9382 internet: http://www.issi.uned.es/ Researchgate: https://www.researchgate.net/profile/Ruben_Heradio

AVISO LEGAL. Este mensaje puede contener información reservada y confidencial. Si usted no es el destinatario no está autorizado a copiar, reproducir o distribuir este mensaje ni su contenido. Si ha recibido este mensaje por error, le rogamos que lo notifique al remitente. Le informamos de que sus datos personales, que puedan constar en este mensaje, serán tratados en calidad de responsable de tratamiento por la UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA (UNED) c/ Bravo Murillo, 38, 28015-MADRID-, con la finalidad de mantener el contacto con usted. La base jurídica que legitima este tratamiento, será su consentimiento, el interés legítimo o la necesidad para gestionar una relación contractual o similar. En cualquier momento podrá ejercer sus derechos de acceso, rectificación, supresión, oposición, limitación al tratamiento o portabilidad de los datos, ante la UNED, Departamento de Política Jurídica de Seguridad de la Informaciónhttps://www.uned.es/dpj, o a través de la Sede electrónicahttps://sede.uned.es/ de la Universidad. Para más información visite nuestra Política de Privacidadhttps://descargas.uned.es/publico/pdf/Politica_privacidad_UNED.pdf.

kuldeepmeel commented 4 years ago

Hi Ruben: So this is going to be a bit tricky to resolve as it seems like a bug with quicksampler (which is not developed by us). But let us give it a try.

Can you follow the following steps and paste the entire output (Every command will generate some output). That would allow us to figure out more information and hopefully the bug as well. 

./samplers/quicksampler -n 5 blasted_case110.cnf
cat blasted_case110.cnf.samples
./samplers/z3 blasted_case110.cnf
cat blasted_case110.cnf.samples.valid
rheradio commented 4 years ago

Dear Meel,

See below the results of running your commands.

Anyway, I was mainly interested in check how barabarik works. From my side, don't worry about the Quicksampler issue.

Thank you so much

Best, Ruben

(base) ruben@ruben-VirtualBox:~/random_sampling/barbarik$ sudo ./samplers/quicksampler -n 5 blasted_case110.cnf [sudo] password for ruben: 01001011001001010 STARTING Samples 2 Execution time 0.0229209 unsat Samples 2 Execution time 0.0240254 Samples 4 Execution time 0.0341094 unsat Samples 4 Execution time 0.0353236 Samples 8 Execution time 0.0432861 Stopping: samples Samples 8 Execution time 0.0433143 Solver time: 0.0374752 Epochs 0, Flips 3, Unsat 2, Calls 6 (base) ruben@ruben-VirtualBox:~/random_sampling/barbarik$ cat blasted_case110.cnf.samples 0: 01001011001001010 1: 11001011001001010 1: 01101011001001001 2: 11101011001001001 1: 01000011001001010 2: 01100011001001001 2: 11000011001001010 3: 11100011001001001 (base) ruben@ruben-VirtualBox:~/random_sampling/barbarik$ sudo ./samplers/z3 blasted_case110.cnf Step 0.000038 s Probability 1.000000 Mutations 0 1 0 1 1 3 0 3 2 3 0 3 3 1 0 1 4 0 0 0 5 0 0 0 6 0 0 0 All 8.000000 / 8 = 1.000000 Valid 0 0 1 8 Invalid Total 0.000507 s Solver 0.000907 s Checked 8 Calls 8 (base) ruben@ruben-VirtualBox:~/random_sampling/barbarik$ cat blasted_case110.cnf.samples.valid 1 1 1 1 1 1 1 1

El lun., 4 may. 2020 a las 21:19, Kuldeep S. Meel (notifications@github.com<mailto:notifications@github.com>) escribió:

Hi Ruben: So this is going to be a bit tricky to resolve as it seems like a bug with quicksampler (which is not developed by us). But let us give it a try.

Can you follow the following steps and paste the entire output (Every command will generate some output). That would allow us to figure out more information and hopefully the bug as well.

./samplers/quicksampler -n 5 blasted_case110.cnf cat blasted_case110.cnf.samples ./samplers/z3 blasted_case110.cnf cat blasted_case110.cnf.samples.valid

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/meelgroup/barbarik/issues/6#issuecomment-623654660, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOLG2YG5GINPYZ6IV3VQ4TRP4INHANCNFSM4LRASJBQ.

--

Prof. Rubén Heradio Gil Universidad Nacional de Educacion a Distancia

Dpto. Ingeniería de Software y Sistemas Informáticos E.T.S.I. Informática, UNED C/ Juan del Rosal, 16 Ciudad Universitaria 28040 Madrid Spain

Phone: (+34) 91 398-8242 e-mail: rheradio@issi.uned.esmailto:rheradio@issi.uned.es Fax : (+34) 91 398-9382 internet: http://www.issi.uned.es/ Researchgate: https://www.researchgate.net/profile/Ruben_Heradio

AVISO LEGAL. Este mensaje puede contener información reservada y confidencial. Si usted no es el destinatario no está autorizado a copiar, reproducir o distribuir este mensaje ni su contenido. Si ha recibido este mensaje por error, le rogamos que lo notifique al remitente. Le informamos de que sus datos personales, que puedan constar en este mensaje, serán tratados en calidad de responsable de tratamiento por la UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA (UNED) c/ Bravo Murillo, 38, 28015-MADRID-, con la finalidad de mantener el contacto con usted. La base jurídica que legitima este tratamiento, será su consentimiento, el interés legítimo o la necesidad para gestionar una relación contractual o similar. En cualquier momento podrá ejercer sus derechos de acceso, rectificación, supresión, oposición, limitación al tratamiento o portabilidad de los datos, ante la UNED, Departamento de Política Jurídica de Seguridad de la Informaciónhttps://www.uned.es/dpj, o a través de la Sede electrónicahttps://sede.uned.es/ de la Universidad. Para más información visite nuestra Política de Privacidadhttps://descargas.uned.es/publico/pdf/Politica_privacidad_UNED.pdf.

kuldeepmeel commented 4 years ago

Okay; so I see something strange.

Can you run the commands without "sudo".

rheradio commented 4 years ago

Hi,

I've realized the root of the problems was directory permissions. Once I've changed the permissions to the user "ruben" to be the owner of the barbarik folder (and its subfolders), everything works fine.

Sorry for the confusion.

Best, Ruben

./barbarik.py --seed 1 --sampler 2 blasted_case110.cnf Experiment: 0 of 1 constantFactor:38 boundFactor: 10.219408528680095 logBoundFactor: 3.3532397943494647 tj: 1789 totalLoops: 2 beta: 0.8181818181818182 epsilon: 0.6 numSolutions: 389 loThresh:56 hiThresh: 332 Using sampler: QuickSampler 11100001111010001 STARTING Samples 2 Execution time 0.0996775 unsat Samples 2 Execution time 0.103445 Samples 4 Execution time 0.138478 unsat Samples 4 Execution time 0.144051 Samples 8 Execution time 0.181324 Stopping: samples Samples 8 Execution time 0.181402 Solver time: 0.155286 Epochs 0, Flips 3, Unsat 2, Calls 6 Step 0.000161 s Probability 1.000000 Mutations 0 1 0 1 1 3 0 3 2 3 0 3 3 1 0 1 4 0 0 0 5 0 0 0 6 0 0 0 All 8.000000 / 8 = 1.000000 Valid 0 0 1 8 Invalid Total 0.002092 s Solver 0.003728 s Checked 8 Calls 8 Number of solutions returned by sampler: 1 Using sampler: SPUR Number of solutions returned by sampler: 1 Using sampler: QuickSampler 11101101111010001010001011111010000110110011100001111010001 STARTING unsat Samples 1 Execution time 0.102079 Samples 2 Execution time 0.142647 Samples 4 Execution time 0.189446 Samples 6 Execution time 0.235971 Samples 12 Execution time 0.269015 Samples 24 Execution time 0.313034 Samples 30 Execution time 0.350631 Samples 60 Execution time 0.384732 Samples 60 Execution time 0.40785 Samples 120 Execution time 0.44235 Samples 192 Execution time 0.473665 Samples 288 Execution time 0.504951 unsat Samples 288 Execution time 0.511533 Samples 564 Execution time 0.545275 Samples 1064 Execution time 0.579925 Samples 1921 Execution time 0.631908 Samples 2778 Execution time 0.681169 Stopping: samples Samples 2778 Execution time 0.681324 Solver time: 0.635667 Epochs 0, Flips 14, Unsat 2, Calls 18 Step 0.000276 s Probability 1.000000 Mutations 0 1 0 1 1 14 0 14 2 78 4 82 3 235 40 275 4 435 147 582 5 541 340 881 6 463 480 943 All 1767.000000 / 2778 = 0.636069 Valid 0 0 1 1767 Invalid 0 0 1 1011 Total 0.558134 s Solver 0.548084 s Checked 2778 Calls 2778 Number of solutions returned by sampler: 389 baseMap: 389 numSolutions: 389 SolutionsCount: 1 loThresh: 56 hiThresh: 332 sampler: QuickSampler i: 1 isUniform: 0 TotalSolutionsGenerated: 390 exp:0 RejectIteration:1 Loop:2 TotalSolutionsGenerated:390 TotalUniformSamples:1

AVISO LEGAL. Este mensaje puede contener información reservada y confidencial. Si usted no es el destinatario no está autorizado a copiar, reproducir o distribuir este mensaje ni su contenido. Si ha recibido este mensaje por error, le rogamos que lo notifique al remitente. Le informamos de que sus datos personales, que puedan constar en este mensaje, serán tratados en calidad de responsable de tratamiento por la UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA (UNED) c/ Bravo Murillo, 38, 28015-MADRID-, con la finalidad de mantener el contacto con usted. La base jurídica que legitima este tratamiento, será su consentimiento, el interés legítimo o la necesidad para gestionar una relación contractual o similar. En cualquier momento podrá ejercer sus derechos de acceso, rectificación, supresión, oposición, limitación al tratamiento o portabilidad de los datos, ante la UNED, Departamento de Política Jurídica de Seguridad de la Informaciónhttps://www.uned.es/dpj, o a través de la Sede electrónicahttps://sede.uned.es/ de la Universidad. Para más información visite nuestra Política de Privacidadhttps://descargas.uned.es/publico/pdf/Politica_privacidad_UNED.pdf.

kuldeepmeel commented 4 years ago

Great; thanks for opening the issue as it allowed us to fix other issues. I am closing this issue but please open another issue if you face difficulty in running barbarik. Looking forward to learning about the usage of Barbarik in your application.