Modified langdetect detect method to decrease run-time
Fixed indentation error in transform_info
Prototyped removal of offset in transform_all_untransformed
This change needs modifications: it fails for the first batch, since the batch is already computed, so that only one ScraperResult.date is greater than or equal to max(batch, key=lambda v: v.date).date, so only one post is transformed.
Originally modified TelegramTelethonTransformer to have a self.client attribute, but this caused the transformer tests to fail, since having a TelegramTelethonScraper already initialized while initializing a TelegramTelethonTransformer causes the Telethon session database to be locked. This could be addressed by deleting the controller object in tests/transformer/telegram_telethon.py
transform_info
transform_all_untransformed
batch
is already computed, so that only oneScraperResult.date
is greater than or equal tomax(batch, key=lambda v: v.date).date
, so only one post is transformed.TelegramTelethonTransformer
to have aself.client
attribute, but this caused the transformer tests to fail, since having aTelegramTelethonScraper
already initialized while initializing aTelegramTelethonTransformer
causes the Telethon session database to be locked. This could be addressed by deleting thecontroller
object intests/transformer/telegram_telethon.py