the code in the book:
import string
samples = ['The cat sat on the mat.', 'The dog ate my homework.']
characters = string.printable
token_index = dict(zip(range(1, len(characters) + 1), characters))
max_length = 50
results = np.zeros((len(samples), max_length, max(token_index.keys()) + 1))
for i, sample in enumerate(samples):
for j, character in enumerate(sample):
index = token_index.get(character)
results[i, j, index] = 1.
the code in github:
import string
samples = ['The cat sat on the mat.', 'The dog ate my homework.']
characters = string.printable # All printable ASCII characters.
token_index = dict(zip(characters, range(1, len(characters) + 1)))
max_length = 50
results = np.zeros((len(samples), max_length, max(token_index.values()) + 1))
for i, sample in enumerate(samples):
for j, character in enumerate(sample[:max_length]):
index = token_index.get(character)
results[i, j, index] = 1.
the code in the book: import string samples = ['The cat sat on the mat.', 'The dog ate my homework.'] characters = string.printable token_index = dict(zip(range(1, len(characters) + 1), characters)) max_length = 50 results = np.zeros((len(samples), max_length, max(token_index.keys()) + 1)) for i, sample in enumerate(samples): for j, character in enumerate(sample): index = token_index.get(character) results[i, j, index] = 1.
the code in github:
import string
samples = ['The cat sat on the mat.', 'The dog ate my homework.'] characters = string.printable # All printable ASCII characters. token_index = dict(zip(characters, range(1, len(characters) + 1)))
max_length = 50 results = np.zeros((len(samples), max_length, max(token_index.values()) + 1)) for i, sample in enumerate(samples): for j, character in enumerate(sample[:max_length]): index = token_index.get(character) results[i, j, index] = 1.