https://www.confluent.io/blog/okay-store-data-apache-kafka/
https://news.ycombinator.com/item?id=15258547
blog-post-ok-to-store-data-in-kafka#the-kappa-architecture1 2When the logic of the stream processing code changes, you often want to recompute your results. A very simple way to do this is just to reset the offset for the program to zero to recompute the results with the new code. This sometimes goes by the somewhat grandiose name of The Kappa Architecture. blog-post-ok-to-store-data-in-kafka#the-kappa-architecture1 2
blog-post-ok-to-store-data-in-kafka#database-inside-out1I think it makes more sense to think of your datacenter as a giant database, in that database Kafka is the commit log, and these various storage systems are kinds of derived indexes or views. It's true that a log like Kafka can be fantastic primitive for building a database, but the queries are still served out of indexes that were built for the appropriate access pattern. blog-post-ok-to-store-data-in-kafka#database-inside-out1