Apache Kafka
Why systems need to send data/messages/logs?
• Loose coupled
• MSA
• Event Process
• Autoscaling - instance termination
• Data Lake
Why systems need to interchange Large Data?
• Realtime Data Processing
• Business Metric
• Service Metric
• Application Log
• System Metric
• Longterm Data Processing
• Realtime Event Triggering
• Separation service instance and data instance
Traditional Data Interchange
Service/Data Hub
Kafka vs MQ
www.slideshare.net/sbaltagi/apache-kafka-vs-rabbitmq-fit-for-purpose-decision-tree
Apache Kafka vs RabbitMQ: Fit For Purpose / Decision Tree
Kafka as a streaming data platform is becoming the successor to traditional messaging systems such as RabbitMQ. Nevertheless, there are still some use cases wh…
www.slideshare.net
Kafka is (vs MQ )
• Topic Pub/sub Model
• Message Replay
• Save messages to disk
• Scalability
• High Throughput
• Asynchronous
• Protocol Free : String data
Kafka Terminology
• Producer/Consumer
• Topic
• Partition
• Read Offset in each partition
• Consumer Group
• Message Broker/Node
• Leader/Follower
• Replication Factor
• ISR (InSyncReplica)
Kafka Architecture
References
www.confluent.io/blog/event-streamingplatform-1/
www.slideshare.net/sbaltagi/apache-fit-for-purpose-decision-tree
[Kafka, 카프카] 아파치 카프카(Apache Kafka) 아키텍처 및 동작방식, 파티션 읽기 쓰기(Partition Read and W
| 카프카(Kafka)란? 아파치 카프카(Apache Kafka)는 분산 스트리밍 플랫폼이며 데이터 파이프 라인을 만들 때 주로 사용되는 오픈소스 솔루션입니다. 카프카는 대용량의 실시간 로그처리에 특화되어
engkimbs.tistory.com
ssup2.github.io/theory_analysis/Kafka_Cluster_Replication/
Kafka Cluster, Replication
Kafka의 Cluster, Replication 기법을 분석한다.
ssup2.github.io