"error: 'i' format requires -2147483648 <= number <= 2147483647"

[bmorris3]

Error of the day:

Constructing model...
Initializing SMC sampler...
Multiprocess sampling (4 chains in 4 jobs)
Running SMC...
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-4-260fdee5be4f> in <module>
     18     print('Running SMC...')
     19     trace_smc = m.sample_smc(draws=draws_smc, random_seed=42, parallel=True, 
---> 20                              cores=4)
     21 
     22     print('Running NUTS...')

~/dot/dot/model.py in sample_smc(self, draws, random_seed, parallel, cores, **kwargs)
    260             with self.pymc_model:
    261                 trace = sample_smc(draws, random_seed=random_seed,
--> 262                                    parallel=parallel, cores=cores, **kwargs)
    263         return trace
    264 

/opt/conda/lib/python3.7/site-packages/pymc3/smc/sample_smc.py in sample_smc(draws, kernel, n_steps, start, tune_steps, p_acc_rate, threshold, save_sim_data, model, random_seed, parallel, chains, cores)
    189         with mp.Pool(cores) as pool:
    190             results = pool.starmap(
--> 191                 sample_smc_int, [(*params, random_seed[i], i, loggers[i]) for i in range(chains)]
    192             )
    193 

/opt/conda/lib/python3.7/multiprocessing/pool.py in starmap(self, func, iterable, chunksize)
    274         `func` and (a, b) becomes func(a, b).
    275         '''
--> 276         return self._map_async(func, iterable, starmapstar, chunksize).get()
    277 
    278     def starmap_async(self, func, iterable, chunksize=None, callback=None,

/opt/conda/lib/python3.7/multiprocessing/pool.py in get(self, timeout)
    655             return self._value
    656         else:
--> 657             raise self._value
    658 
    659     def _set(self, i, obj):

/opt/conda/lib/python3.7/multiprocessing/pool.py in _handle_tasks(taskqueue, put, outqueue, pool, cache)
    429                         break
    430                     try:
--> 431                         put(task)
    432                     except Exception as e:
    433                         job, idx = task[:2]

/opt/conda/lib/python3.7/multiprocessing/connection.py in send(self, obj)
    204         self._check_closed()
    205         self._check_writable()
--> 206         self._send_bytes(_ForkingPickler.dumps(obj))
    207 
    208     def recv_bytes(self, maxlength=None):

/opt/conda/lib/python3.7/multiprocessing/connection.py in _send_bytes(self, buf)
    391         n = len(buf)
    392         # For wire compatibility with 3.2 and lower
--> 393         header = struct.pack("!i", n)
    394         if n > 16384:
    395             # The payload is large so Nagle's algorithm won't be triggered

error: 'i' format requires -2147483648 <= number <= 2147483647

Seems to be related to pickling large objects. Might be relevant discussion here.

[bmorris3]

Changing the data from float64 to float32 doesn't help. Shortening the data set may help.

[bmorris3]

Activating partition_lons seems to solve the problem after introducing an absolute lower radius bound of 1e-6.

[bmorris3]

Summary of diagnostic runs:

Number of LC points	Success/fail
4000	Fail
2000	Success
1300	Success