HostBuffer.h File Reference

HostBuffer - a buffer that contains a shared or private bump pool to either externally or internally managed host memory. More...

#include <nanovdb/NanoVDB.h>
#include <stdint.h>
#include <cstdio>
#include <cstdlib>
#include <memory>
#include <mutex>
#include <unordered_set>
#include <cassert>
#include <sstream>
#include <cstring>

Go to the source code of this file.

Classes
struct	BufferTraits< BufferT >
class	HostBuffer
	This is a buffer that contains a shared or private pool to either externally or internally managed host memory. More...
struct	HostBuffer::Pool

Namespaces
	nanovdb

Macros
#define	checkPtr(ptr, msg)

Detailed Description

HostBuffer - a buffer that contains a shared or private bump pool to either externally or internally managed host memory.

Date

April 20, 2021

This HostBuffer can be used in multiple ways, most of which are demonstrated in the examples below. Memory in the pool can be managed or unmanged (e.g. internal or external) and can be shared between multiple buffers or belong to a single buffer.

Example that uses HostBuffer::create inside io::readGrids to create a full self-managed buffer, i.e. not shared and without padding, per grid in the file.

auto handles = nanovdb::io::readGrids("file.nvdb");

Example that uses HostBuffer::createFull. Assuming you have a raw pointer to a NanoVDB grid of unknown type, this examples shows how to create its GridHandle which can be used to enquire about the grid type and meta data.

void    *data;// pointer to a NanoVDB grid of unknown type
uint64_t size;// byte size of NanoVDB grid of unknown type
auto buffer = nanovdb::HostBuffer::createFull(size, data);
nanovdb::GridHandle<> gridHandle(std::move(buffer));

Example that uses HostBuffer::createPool for internally managed host memory. Suppose you want to read multiple grids in multiple files, but reuse the same fixed sized memory buffer to both avoid memory fragmentation as well as exceeding the fixed memory ceiling!

auto pool = nanovdb::HostBuffer::createPool(1 << 30);// 1 GB memory pool
std::vector<std::string>> frames;// vector of grid names
for (int i=0; i<frames.size(); ++i) {
    auto handles = nanovdb::io::readGrids(frames[i], 0, pool);// throws if grids in file exceed 1 GB
    ...
    pool.reset();// clears all handles and resets the memory pool for reuse
}

Example that uses HostBuffer::createPool for externally managed host memory. Note that in this example handles are allowed to outlive pool since they internally store a shared pointer to the memory pool. However data MUST outlive handles since the pool does not own its memory in this example.

const size_t poolSize = 1 << 30;// 1 GB
uint8_t *data = static_cast<uint8_t*>(std::malloc(size)+NANOVDB_DATA_ALIGNMENT);// 1 GB pool
uint8_t *buffer = nanovdb::alignPtr(data);// 32B aligned buffer
//uint8_t *buffer = std::aligned_alloc(NANOVDB_DATA_ALIGNMENT, poolSize);// in C++17
auto pool = nanovdb::HostBuffer::createPool(poolSize, buffer);
auto handles1 = nanovdb::io::readGrids("file1.nvdb", 0, pool);
auto handles2 = nanovdb::io::readGrids("file2.nvdb", 0, pool);
....
std::free(data);

Example that uses HostBuffer::createPool for externally managed host memory. Note that in this example handles are allowed to outlive pool since they internally store a shared pointer to the memory pool. However array MUST outlive handles since the pool does not own its memory in this example.

const size_t poolSize = 1 << 30;// 1 GB
std::unique_ptr<uint8_t[]> array(new uint8_t[size+NANOVDB_DATA_ALIGNMENT]);// scoped pool of 1 GB
//std::unique_ptr<uint8_t[]> array(std::aligned_alloc(NANOVDB_DATA_ALIGNMENT, size));// in C++17
uint8_t *buffer = nanovdb::alignPtr(array.get());// 32B aligned buffer
auto pool = nanovdb::HostBuffer::createPool(poolSize, buffer);
auto handles = nanovdb::io::readGrids("file.nvdb", 0, pool);

Macro Definition Documentation

#define checkPtr	(		ptr,
			msg
	)

Value:

{ \
        ptrAssert((ptr), (msg), __FILE__, __LINE__); \
    }