I am trying to run the network using the "Amazon Review Polarity" database, but Python can't manage to create a cubic matrix having (400000, 1014, 69) dimensions, as it throws a MemoryError in this case. But only when I changed it from np.zeros((len(x), maxlen, vocab_size)) to np.zeros((len(x), maxlen, vocab_size), dtype=np.uint8) Python managed to create the cubic matrix, although it used more than 20 GB though.
Taking the fact that the matrix is a one-hot encoding representation, it only needs 0s and 1s and not float numbers (as it was being created before), can I safely assure that it will work correctly? I mean, I just changed the type from float to a smaller int, am I missing something?
I am trying to run the network using the "Amazon Review Polarity" database, but Python can't manage to create a cubic matrix having (400000, 1014, 69) dimensions, as it throws a MemoryError in this case. But only when I changed it from
np.zeros((len(x), maxlen, vocab_size))tonp.zeros((len(x), maxlen, vocab_size), dtype=np.uint8)Python managed to create the cubic matrix, although it used more than 20 GB though.Taking the fact that the matrix is a one-hot encoding representation, it only needs 0s and 1s and not float numbers (as it was being created before), can I safely assure that it will work correctly? I mean, I just changed the type from float to a smaller int, am I missing something?