In August I gave a talk at the Atlanta Go User Group related to Apache Beam processing using Go.
Go is fast, but what happens when your data is larger than what a single process can handle within your available time window? Especially if you need to calculate metrics related to the data across those machines. That’s where Golang, Apache Beam and Dataflow come in.
Apache Beam is an open source, unified model for defining both batch and streaming data-parallel processing pipelines. https://beam.apache.org/get-started/beam-overview/ Dataflow is a unified stream and batch data processing service on GCP that’s serverless, fast, and cost-effective. https://cloud.google.com/dataflow
Presentation: https://docs.google.com/presentation/d/13Uk2UBShPLWGNDKQoFkd2vMPRLyeyAUAL7G1JBLt6Ak/
Go User Group Page here: https://www.meetup.com/go-users-group-atlanta/events/294740555/