Open zzt93 opened 6 years ago
clone
& dup
semantic -- change to create
StandardEvaluationContext
(20% memory reduction)upsert
for es output channel if 404key
SyncData
SyncUtil
as separate jar to maven centralinput:
masters:
- connection:
address: ${HOST_ADDRESS}
port: 27018
type: Mongo
repos:
- name: "chat"
entities:
- name: messages
fields: [time, content]
syncer-data
deployconfig
packageif
and/or enhance switcher
JavaCompiler
to generate code dynamically rather than spring ellower-hyphen
SyncData
: update event
should have before
& fields
data:
updated()
& udpated(String name)
method for usebefore
to get before datasyncer-consumerId-filter-1
-
FileBasedMap
record last removed position if map is emptynested
objat-least-once-semantic
, not do._all
Update position even not interested in
Share storage in k8sMode
/instanceId/syncer/xx
Kafka output: timestamp to long;
Mysql output: auto add id;
[Impl Pending] Update sync meta position when consumer not interested in this event?
num
not interested event happenedETL
non-block-mode
TODO
Cold-start (ETL) opt
Test Framework more
Add
/stat
,/input
endpoint for syncerTimezone config
Convert MySQL integer as byte array
Dependency module: not package & load if not use mongo sync etc.
IncludeBefore & IncludeUpdated config?
rpm
anddpkg
Sync check: query input & output for comparing
SyncData
?Warning if multiple
schema.table
has different rowsA better serializer than json, which lost the type info: PB?
Avro?Test redis output & nested sql
Config thread for each consumer
Support set parent of ES
Row image format support?
Batch module & failure module is coupled with channel module
MDC.put eventId is necessary??