Mongodb – Understanding how journal mode works with j:true

One of things that has recently confused me with Mongodb while using Mongoid with the Moped driver is the use of the journal mode and the j:1 write concern.

Journal Mode

The journal mode ensures that your writes are written to a redo/write ahead log after its written to the memory but before its pushed to disk. It provides a level of durability where if the server were to crash before certain data is written to disk, on restart this data is read from the journal logs and written to disk. Thus making your data more resilient.

Journaling is enabled on all 64 bit installations since 2.0 as a default but not on the 32 bit ones.

What kind of stuff does journaling record

Journaling takes care of

  • Create/update commands
  • Index creation
  • namespace changes.

Journal Commit Interval

The journal commit interval (journalCommitInterval) is the time interval between subsequent writes to the journal files and is set 100 ms as a default. This is true if your journal files and data file are on the same server. Its 30 ms if they aren’t on the same machine.

This means you can lose data that has not yet been journaled in the last 100ms. You can update the journalCommitInternval setting in your mongodb.conf file to anywhere between 2ms to 300ms. A lower value will increase the frequency with which data is journaled but at the cost of disk performance.

To force mongodb to commit sooner to the journal without updating the journalCommitInterval setting one could use the j:1 write concern. As per the docs

When a write operation with j:true is pending, mongod will reduce journalCommitInterval to a third of the set value.

So a low (whatever works best for you) journalCommitInterval with saves called with j:1 would ensure that the system remains consistent.

To verify that your data was written to the journal you could call the

Most drivers run this for you so this isn’t something that you need to worry about.

Mongoid

The mongoid documentation for versions 3.0 and above does not talk about the j:1 write concern for some reason but here is what i found based on this discussion.

This is equivalent to calling it with

This implies that the write will be acknowledged once the data is written to the primary’s memory but not to disk. If you want to trigger a journal write as well then you would have to do it with

Most of the data was collected and posted as an answer on stackoverflow by mnemosyn and is available on the mongodb docs. My goal was to create a quick reference.

References:

  1. http://docs.mongodb.org/manual/reference/write-concern/
  2. http://docs.mongodb.org/manual/core/journaling/
  3. http://docs.mongodb.org/manual/tutorial/manage-journaling/
  4. http://docs.mongodb.org/manual/reference/configuration-options/#journalCommitInterval
  5. http://docs.mongodb.org/manual/core/journaling/
  6. http://docs.mongodb.org/manual/core/write-concern/#write-concern
  7. http://rsmith.co/2012/11/05/mongodb-gotchas-and-how-to-avoid-them/
  8. https://groups.google.com/forum/#!msg/mongoid/Bybccfc0gLI/Aei0VkMAZqoJ
Advertisements