search

ziyin-dl / word-embedding-dimensionality-selection

On the Dimensionality of Word Embedding
https://nips.cc/Conferences/2018/Schedule?showEvent=12567
MIT License
329 stars 44 forks source link

core dumped error #18

Open shenxuhui opened 4 years ago

shenxuhui commented 4 years ago

error message:

Segmentation fault (core dumped) nohup python -m main --file data/train.txt --config_file config/train.yml --algorithm word2vec

shenxuhui commented 4 years ago

demo data can run successfully in same environment.

My config file is:

skip_window: 5 neg_samples: 1 vocabulary_size: 94418 min_count: 1

shenxuhui commented 4 years ago

My train_data: 539 M txt file. system: Ubuntu 14.04 hardware: 50 cores and 256GB RAM

shenxuhui commented 4 years ago

Program has run two days before Segmentation fault.

ziyin-dl commented 4 years ago

That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size?

On Wed, Nov 20, 2019, 9:11 PM SXH notifications@github.com wrote:

Program has run two days before Segmentation fault.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ziyin-dl/word-embedding-dimensionality-selection/issues/18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A .

ziyin-dl commented 4 years ago

What was the RAM usage when it occurred?

On Thu, Nov 21, 2019, 7:16 PM Zi Yin s0960974@gmail.com wrote:

That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size?

On Wed, Nov 20, 2019, 9:11 PM SXH notifications@github.com wrote:

Program has run two days before Segmentation fault.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ziyin-dl/word-embedding-dimensionality-selection/issues/18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A .

shenxuhui commented 4 years ago

What was the RAM usage when it occurred? … On Thu, Nov 21, 2019, 7:16 PM Zi Yin @.> wrote: That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size? On Wed, Nov 20, 2019, 9:11 PM SXH @.> wrote: > Program has run two days before Segmentation fault. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A > . >

Thanks for reply, I think the usage of RAM has two stags. The first stage used around 240 GB RAM, after that, the two stage used around 180GB RAM, then the program raise segment fault.