fire::io::Writer::Buffer< AtomicType > Class Template Reference

Buffer atomic data types in-memory for efficient disk operations. More...

Inheritance diagram for fire::io::Writer::Buffer< AtomicType >:

Collaboration diagram for fire::io::Writer::Buffer< AtomicType >:

Public Member Functions
	Buffer (std::size_t max, HighFive::DataSet s)
	Define the buffer size and the set we will write to. More...
virtual	~Buffer ()=default
	destruct the in-memory buffer
void	save (const AtomicType &val)
	Put the new value into the buffer. More...
virtual void	flush () final override
	Flush our in-memory buffer onto disk. More...
Public Member Functions inherited from fire::io::Writer::BufferHandle
	BufferHandle (std::size_t max, HighFive::DataSet s)
	Define the maximum size of the buffer and the dataset we are writing to. More...
virtual	~BufferHandle ()=default
	virtual destructor so derived Buffer can be destructed properly

Private Attributes
std::vector< AtomicType >	buffer_
	the actual buffer of data in-memory
std::size_t	i_file_
	the index of the file we will write to on the next flush

Additional Inherited Members
Protected Attributes inherited from fire::io::Writer::BufferHandle
std::size_t	max_len_
	the maximum size of the buffer
HighFive::DataSet	set_
	the H5 dataset we are writing to

Detailed Description

template<typename AtomicType>
class fire::io::Writer::Buffer< AtomicType >

Buffer atomic data types in-memory for efficient disk operations.

Template Parameters

AtomicType

the data type we are buffering

Constructor & Destructor Documentation

Buffer()

template<typename AtomicType >

fire::io::Writer::Buffer< AtomicType >::Buffer ( std::size_t  max, HighFive::DataSet  s  )

inlineexplicit

Define the buffer size and the set we will write to.

We also use std::vector::reserve to let the memory handler know the size of our buffer. This can help us avoid unnecessary copying and reallocation while using std::vector::push_back to insert elements into the vector.

Parameters

[in]	max	buffer size
[in]	s	dataset to write to

Member Function Documentation

flush()

template<typename AtomicType >

virtual void fire::io::Writer::Buffer< AtomicType >::flush ( )

inlinefinaloverridevirtual

Flush our in-memory buffer onto disk.

We leave early if the buffer is empty. This is helpful for the case where the number of elements in the dataset happen to be an exact multiple of the buffer size. Then the buffer would be empty at the time that Writer::~Writer is called which calls all Buffers to flush in order to avoid data loss.

We determine the new extent of the dataset given how many elements are in the buffer, then we resize the dataset that is on disk to this new extent.

Then, we copy the buffer into the DataSet on disk using a compile-time choice that handles the std::vector<bool> specialization bug in HighFive. and translates bools into our custom enum fire::io::Bool which mimics the serialization behavior of the bool type understandable by h5py.

Finally, we update the file index, clear the buffer, and re-reserve the maximum length of the buffer to prepare for another chunk of data.

Exceptions

HighFive::DataSetException

if unable to extend or write to the DataSet.

Implements fire::io::Writer::BufferHandle.

save()

template<typename AtomicType >

void fire::io::Writer::Buffer< AtomicType >::save ( const AtomicType &  val )

inline

Put the new value into the buffer.

If the buffer goes over the maximum length of the buffer, then we call Buffer::flush

Parameters

[in]

val

data to append to the dataset

---

The documentation for this class was generated from the following file:

• include/fire/io/Writer.h