https://www.youtube.com/watch?v=NVl9_6J1G60
At 4:15 thehe state of the art is to come up with a "tracking plan"
At 12:10 push down join versus distributed joinat 18 he talks about Marshall indexes partial indexes
At 20 a partial index is paid for every single thing you want to track it. The different things are extremely sparse.
At 22 For final analysis it is better if you write a UDF
At 20 for some very serious SQL used for final analysis
At 2730 you can create the next currently currently
talk-powering-heap#idempotent-and-commutativeAt 3020 messages should be idempotent and commutative talk-powering-heap#idempotent-and-commutative
At 52 there is in Kafka a single master that things go through. My question at this point is this cough that use something like the raft consensus algorithm to ensure log replication
talk-powering-heap#commit-logUse it as a commit log for the stuff coming in talk-powering-heap#commit-log
talk-powering-heap#denormalized-and-mutated-event-data1 At 39:15 talks about how events in the postgres shards are somewhat denormalized for speed. Some of the data is added to events after the fact. For example what page the person is on. It needs to be attached to the event so that the partial index can work. This is very interesting. It means that although the events Kafka are immutable and append-only, in Postgres they are derived and mutable. talk-powering-heap#denormalized-and-mutated-event-data1
talk-powering-heap#slow-worse-than-dead1 2 At 43:30, often in distributed systems of having something be slow is worse than it being dead. talk-powering-heap#slow-worse-than-dead1 2
talk-powering-heap#unique-handle1 2 At 44:10 talks about a unique handle that can be used to merge user records. talk-powering-heap#unique-handle1 2
talk-powering-heap#time-tiering1At 4515 they talk about time tiering. Basically you have high medium and cold, like the last two weeks of data you are always wanting to query against. That should be in superfast stuff talk-powering-heap#time-tiering1