search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.29k stars 1.61k forks source link

Python int too large to convert to C long #15925

Closed shuo1990 closed 3 years ago

shuo1990 commented 3 years ago

Description

What steps will reproduce the problem?

Dear sir/madam,

Hi,

My problem is as followed:

a=10000

param_list = list(ParameterSampler(param_grid, n_iter=a,random_state=0)) Traceback (most recent call last):

File "", line 1, in param_list = list(ParameterSampler(param_grid, n_iter=a,random_state=0))

File "C:\Users\Shuo Zhang\anaconda3\lib\site-packages\sklearn\model_selection_search.py", line 284, in iter for i in sample_without_replacement(grid_size, n_iter,

File "sklearn\utils_random.pyx", line 226, in sklearn.utils._random.sample_without_replacement

OverflowError: Python int too large to convert to C long

Could you help me with this issue?

Kind regards,

Shuo

Versions

Dependencies


# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.6.0 (OK)
diff_match_patch >=20181111    :  20200713 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.19.0 (OK)
jedi =0.17.1                   :  0.17.1 (OK)
nbconvert >=4.0                :  6.0.7 (OK)
numpydoc >=0.6.0               :  1.1.0 (OK)
paramiko >=2.4.0               :  2.7.2 (OK)
parso =0.7.0                   :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.2 (OK)
pygments >=2.0                 :  2.7.2 (OK)
pylint >=1.0                   :  2.6.0 (OK)
pyls >=0.34.0;<1.0.0           :  0.35.1 (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  1.0.1 (OK)
qtconsole >=4.6.0              :  4.7.7 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
rtree >=0.8.3                  :  0.9.4 (OK)
sphinx >=0.6.6                 :  3.2.1 (OK)
spyder_kernels >=1.9.4;<1.10.0 :  1.9.4 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.2 (OK)

# Optional:
cython >=0.21                  :  0.29.14 (OK)
matplotlib >=2.0.0             :  3.3.2 (OK)
numpy >=1.7                    :  1.19.5 (OK)
pandas >=0.13.1                :  1.2.5 (OK)
scipy >=0.17.0                 :  1.5.2 (OK)
sympy >=0.7.3                  :  1.6.2 (OK)
dalthviz commented 3 years ago

Hi @shuo1990 this seems unrelated with Spyder. A quick search of the exception message suggest that it could be related with the usage of numbers greater than sys.maxsize. You can check more info here: https://stackoverflow.com/a/38314163/15954282

There is not much from our side to help you but hope the info above helps :)

shuo1990 commented 3 years ago

Hi, Daniel.

Thank you for your quick reply.

For this error, I still cant solve it.

The code can be operated in Colab, but not achieved in spyder because of the reported error: 'Python int too large...'.

Kind regards,

Shuo

This email originated from TU Dublin. If you received this email in error, please delete it from your system. Please note that if you are not the named addressee, disclosing, copying, distributing or taking any action based on the contents of this email or attachments is prohibited.

Is ó OT Baile Átha Cliath a tháinig an ríomhphost seo. Má fuair tú an ríomhphost seo trí earráid, scrios de do chóras é le do thoil. Tabhair ar aird, mura tú an seolaí ainmnithe, go bhfuil dianchosc ar aon nochtadh, aon chóipeáil, aon dáileadh nó ar aon ghníomh a dhéanfar bunaithe ar an ábhar atá sa ríomhphost nó sna hiatáin seo.

From: Daniel Althviz Moré @.> Sent: Wednesday, June 23, 2021 5:37 PM To: spyder-ide/spyder @.> Cc: D18128381 Shuo Zhang @.>; Mention @.> Subject: Re: [spyder-ide/spyder] Python int too large to convert to C long (#15925)

Hi @shuo1990https://github.com/shuo1990 this seems unrelated with Spyder. A quick search of the exception message suggest that it could be related with the usage of numbers greater than sys.maxsize. You can check more info here: https://stackoverflow.com/a/38314163/15954282

There is not much from our side to help you but hope the info above helps :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/15925#issuecomment-866992282, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASWLIYECAONFHJA7SXKEMYLTUIEUPANCNFSM47FYWVYA.

dalthviz commented 3 years ago

Maybe try to check the value of sys.maxsize in Colab and compare it with the one you have in the machine you are running Spyder. let us know!

shuo1990 commented 3 years ago

Hi, Daniel.

The max size of colab is 9223372036854775807.

For spyder is also 9223372036854775807.

Kind regards,

Shuo

This email originated from TU Dublin. If you received this email in error, please delete it from your system. Please note that if you are not the named addressee, disclosing, copying, distributing or taking any action based on the contents of this email or attachments is prohibited.

Is ó OT Baile Átha Cliath a tháinig an ríomhphost seo. Má fuair tú an ríomhphost seo trí earráid, scrios de do chóras é le do thoil. Tabhair ar aird, mura tú an seolaí ainmnithe, go bhfuil dianchosc ar aon nochtadh, aon chóipeáil, aon dáileadh nó ar aon ghníomh a dhéanfar bunaithe ar an ábhar atá sa ríomhphost nó sna hiatáin seo.

From: Daniel Althviz Moré @.> Sent: Wednesday, June 23, 2021 6:40 PM To: spyder-ide/spyder @.> Cc: D18128381 Shuo Zhang @.>; Mention @.> Subject: Re: [spyder-ide/spyder] Python int too large to convert to C long (#15925)

Maybe try to check the value of sys.maxsize in Colab and compare it with the one you have in the machine you are running Spyder. let us know!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/15925#issuecomment-867033494, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASWLIYAZFE3UOJFN3MOYCKLTUIMBXANCNFSM47FYWVYA.

dalthviz commented 3 years ago

Thank you for the info @shuo1990 then seems like something else could be happening. Could you send us a full code example to reproduce it in our side? Let us know

dalthviz commented 3 years ago

Closing due to lack of response

Lee706 commented 1 year ago

Hello Professor! I have also encountered the same problem. May I ask what the solution is? thank you very much

Lee706 commented 1 year ago

OverflowError Traceback (most recent call last)

in () 1 #调整超参数 2 data_df = data_df.astype('float64') ----> 3 tuned_parameters = hyperparameter_tuning(model_type, x_train, y_train, num_of_folds, scaler_option, n_iter=1, random_state=0, verbose=1) in hyperparameter_tuning(tag, x_train, y_train, num_of_folds, scaler_option, n_iter, random_state, verbose) 997 y_train_ = y_train.reshape(y_train.shape[0],) 998 --> 999 model.fit(x_train_, y_train_) 1000 1001 tuned_parameters = add_key_to_params(tag, model.best_params_) C:\Users\Lee\.conda\envs\py36\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs) 61 extra_args = len(args) - len(all_args) 62 if extra_args <= 0: ---> 63 return f(*args, **kwargs) 64 65 # extra_args > 0 C:\Users\Lee\.conda\envs\py36\lib\site-packages\sklearn\model_selection\_search.py in fit(self, X, y, groups, **fit_params) 839 return results 840 --> 841 self._run_search(evaluate_candidates) 842 843 # multimetric is determined here because in the case of a callable C:\Users\Lee\.conda\envs\py36\lib\site-packages\sklearn\model_selection\_search.py in _run_search(self, evaluate_candidates) 1633 evaluate_candidates(ParameterSampler( 1634 self.param_distributions, self.n_iter, -> 1635 random_state=self.random_state)) C:\Users\Lee\.conda\envs\py36\lib\site-packages\sklearn\model_selection\_search.py in evaluate_candidates(candidate_params, cv, more_results) 785 more_results=None): 786 cv = cv or cv_orig --> 787 candidate_params = list(candidate_params) 788 n_candidates = len(candidate_params) 789 C:\Users\Lee\.conda\envs\py36\lib\site-packages\sklearn\model_selection\_search.py in __iter__(self) 290 n_iter = grid_size 291 for i in sample_without_replacement(grid_size, n_iter, --> 292 random_state=rng): 293 yield param_grid[i] 294 sklearn\utils\_random.pyx in sklearn.utils._random.sample_without_replacement() OverflowError: Python int too large to convert to C long