Java/JVM and Software Design Brainstorming

In this post we will share a starting project to use Apache ignite data grid an event and snapshot store to mix the benefits of the event sourcing and the data grid .

The implementation is based into the Journal plugin TCK specs provided by Akka persistence.

This is mainly using Apache ignite with akka persistence to provide journal and snapshot store by using the partitioned caches and benefit from the distributed highly available data grid features plus the nice query and data computations features in Ignite that can be used to have normalized views from the event store and do analytical jobs over them despite it is advised to keep write nodes separate from read nodes for better scalability.

Akka and Ignite used versions:

Akka version :2.5.7+ , Ignite Version :2.3.0+

Journal plugin

• All operations required by the Akka Persistence journal plugin API are fully supported.
• It use apache ignite partitioned cache with default number of backups to 1 , that can be changed into reference.conf file.

Snapshot store plugin

• Implements the Akka Persistence snapshot store plugin API.

How to use

Enable the plugins into your akka cluster configuration:

akka.persistence.journal.plugin = "akka.persistence.journal.ignite"
akka.persistence.snapshot-store.plugin = "akka.persistence.snapshot.ignite"

Configure Ignite data grid properties , default configured on localhost.

ignite {
  //to start client or server node to connect to Ignite data cluster 
  isClientNode = false
  // for ONLY testing we use localhost
  // used for grid cluster connectivity
  tcpDiscoveryAddresses = "localhost"
  metricsLogFrequency = 0
  // thread pools used by Ignite , should based into target machine specs
  queryThreadPoolSize = 4
  dataStreamerThreadPoolSize = 1
  managementThreadPoolSize = 2
  publicThreadPoolSize = 4
  systemThreadPoolSize = 2
  rebalanceThreadPoolSize = 1
  asyncCallbackPoolSize = 4
  peerClassLoadingEnabled = false
  // to enable or disable durable memory persistance
  enableFilePersistence = true
  // used for grid cluster connectivity, change it to suit your configuration 
  igniteConnectorPort = 11211
  // used for grid cluster connectivity , change it to suit your configuration 
  igniteServerPortRange = "47500..47509"
  //durable memory persistance storage file system path , change it to suit your configuration 
  ignitePersistenceFilePath = "./data"
}

 

and you will have ignite enabled as your journal and snapshot plugins , you can enable it by starting server node or client based into the configuration  above .

Technical details :

the main journal implementation is IgniteWriteJournal :

the main snapshot implementation class is IgniteSnapshotStore  :

For more details feel free to dive into the code based , it is a small code base for now !.

Summary :

• This is working in progress any contribution would be really helpful , please check the project on GitHub and give a hand !
  • project in GitHub : https://github.com/Romeh/akka-persistance-ignite