OpenVDB  11.0.0
NanoVDB.h
Go to the documentation of this file.
1 // Copyright Contributors to the OpenVDB Project
2 // SPDX-License-Identifier: MPL-2.0
3 
4 /*!
5  \file NanoVDB.h
6 
7  \author Ken Museth
8 
9  \date January 8, 2020
10 
11  \brief Implements a light-weight self-contained VDB data-structure in a
12  single file! In other words, this is a significantly watered-down
13  version of the OpenVDB implementation, with few dependencies - so
14  a one-stop-shop for a minimalistic VDB data structure that run on
15  most platforms!
16 
17  \note It is important to note that NanoVDB (by design) is a read-only
18  sparse GPU (and CPU) friendly data structure intended for applications
19  like rendering and collision detection. As such it obviously lacks
20  a lot of the functionality and features of OpenVDB grids. NanoVDB
21  is essentially a compact linearized (or serialized) representation of
22  an OpenVDB tree with getValue methods only. For best performance use
23  the ReadAccessor::getValue method as opposed to the Tree::getValue
24  method. Note that since a ReadAccessor caches previous access patterns
25  it is by design not thread-safe, so use one instantiation per thread
26  (it is very light-weight). Also, it is not safe to copy accessors between
27  the GPU and CPU! In fact, client code should only interface
28  with the API of the Grid class (all other nodes of the NanoVDB data
29  structure can safely be ignored by most client codes)!
30 
31 
32  \warning NanoVDB grids can only be constructed via tools like createNanoGrid
33  or the GridBuilder. This explains why none of the grid nodes defined below
34  have public constructors or destructors.
35 
36  \details Please see the following paper for more details on the data structure:
37  K. Museth, “VDB: High-Resolution Sparse Volumes with Dynamic Topology”,
38  ACM Transactions on Graphics 32(3), 2013, which can be found here:
39  http://www.museth.org/Ken/Publications_files/Museth_TOG13.pdf
40 
41  NanoVDB was first published there: https://dl.acm.org/doi/fullHtml/10.1145/3450623.3464653
42 
43 
44  Overview: This file implements the following fundamental class that when combined
45  forms the backbone of the VDB tree data structure:
46 
47  Coord- a signed integer coordinate
48  Vec3 - a 3D vector
49  Vec4 - a 4D vector
50  BBox - a bounding box
51  Mask - a bitmask essential to the non-root tree nodes
52  Map - an affine coordinate transformation
53  Grid - contains a Tree and a map for world<->index transformations. Use
54  this class as the main API with client code!
55  Tree - contains a RootNode and getValue methods that should only be used for debugging
56  RootNode - the top-level node of the VDB data structure
57  InternalNode - the internal nodes of the VDB data structure
58  LeafNode - the lowest level tree nodes that encode voxel values and state
59  ReadAccessor - implements accelerated random access operations
60 
61  Semantics: A VDB data structure encodes values and (binary) states associated with
62  signed integer coordinates. Values encoded at the leaf node level are
63  denoted voxel values, and values associated with other tree nodes are referred
64  to as tile values, which by design cover a larger coordinate index domain.
65 
66 
67  Memory layout:
68 
69  It's important to emphasize that all the grid data (defined below) are explicitly 32 byte
70  aligned, which implies that any memory buffer that contains a NanoVDB grid must also be at
71  32 byte aligned. That is, the memory address of the beginning of a buffer (see ascii diagram below)
72  must be divisible by 32, i.e. uintptr_t(&buffer)%32 == 0! If this is not the case, the C++ standard
73  says the behaviour is undefined! Normally this is not a concerns on GPUs, because they use 256 byte
74  aligned allocations, but the same cannot be said about the CPU.
75 
76  GridData is always at the very beginning of the buffer immediately followed by TreeData!
77  The remaining nodes and blind-data are allowed to be scattered throughout the buffer,
78  though in practice they are arranged as:
79 
80  GridData: 672 bytes (e.g. magic, checksum, major, flags, index, count, size, name, map, world bbox, voxel size, class, type, offset, count)
81 
82  TreeData: 64 bytes (node counts and byte offsets)
83 
84  ... optional padding ...
85 
86  RootData: size depends on ValueType (index bbox, voxel count, tile count, min/max/avg/standard deviation)
87 
88  Array of: RootData::Tile
89 
90  ... optional padding ...
91 
92  Array of: Upper InternalNodes of size 32^3: bbox, two bit masks, 32768 tile values, and min/max/avg/standard deviation values
93 
94  ... optional padding ...
95 
96  Array of: Lower InternalNodes of size 16^3: bbox, two bit masks, 4096 tile values, and min/max/avg/standard deviation values
97 
98  ... optional padding ...
99 
100  Array of: LeafNodes of size 8^3: bbox, bit masks, 512 voxel values, and min/max/avg/standard deviation values
101 
102 
103  Notation: "]---[" implies it has optional padding, and "][" implies zero padding
104 
105  [GridData(672B)][TreeData(64B)]---[RootData][N x Root::Tile]---[InternalData<5>]---[InternalData<4>]---[LeafData<3>]---[BLINDMETA...]---[BLIND0]---[BLIND1]---etc.
106  ^ ^ ^ ^ ^ ^
107  | | | | | |
108  +-- Start of 32B aligned buffer | | | | +-- Node0::DataType* leafData
109  GridType::DataType* gridData | | | |
110  | | | +-- Node1::DataType* lowerData
111  RootType::DataType* rootData --+ | |
112  | +-- Node2::DataType* upperData
113  |
114  +-- RootType::DataType::Tile* tile
115 
116 */
117 
118 #ifndef NANOVDB_NANOVDB_H_HAS_BEEN_INCLUDED
119 #define NANOVDB_NANOVDB_H_HAS_BEEN_INCLUDED
120 
121 // NANOVDB_MAGIC_NUMBER is currently used for both grids and files (starting with v32.6.0)
122 // NANOVDB_MAGIC_GRID will soon be used exclusively for grids
123 // NANOVDB_MAGIC_FILE will soon be used exclusively for files
124 // NANOVDB_MAGIC_NODE will soon be used exclusively for NodeManager
125 // | : 0 in 30 corresponds to 0 in NanoVDB0
126 #define NANOVDB_MAGIC_NUMBER 0x304244566f6e614eUL // "NanoVDB0" in hex - little endian (uint64_t)
127 #define NANOVDB_MAGIC_GRID 0x314244566f6e614eUL // "NanoVDB1" in hex - little endian (uint64_t)
128 #define NANOVDB_MAGIC_FILE 0x324244566f6e614eUL // "NanoVDB2" in hex - little endian (uint64_t)
129 #define NANOVDB_MAGIC_NODE 0x334244566f6e614eUL // "NanoVDB3" in hex - little endian (uint64_t)
130 #define NANOVDB_MAGIC_MASK 0x00FFFFFFFFFFFFFFUL // use this mask to remove the number
131 //#define NANOVDB_USE_NEW_MAGIC_NUMBERS// used to enable use of the new magic numbers described above
132 
133 #define NANOVDB_MAJOR_VERSION_NUMBER 32 // reflects changes to the ABI and hence also the file format
134 #define NANOVDB_MINOR_VERSION_NUMBER 6 // reflects changes to the API but not ABI
135 #define NANOVDB_PATCH_VERSION_NUMBER 0 // reflects changes that does not affect the ABI or API
136 
137 #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
138 
139 // This replaces a Coord key at the root level with a single uint64_t
140 #define NANOVDB_USE_SINGLE_ROOT_KEY
141 
142 // This replaces three levels of Coord keys in the ReadAccessor with one Coord
143 //#define NANOVDB_USE_SINGLE_ACCESSOR_KEY
144 
145 // Use this to switch between std::ofstream or FILE implementations
146 //#define NANOVDB_USE_IOSTREAMS
147 
148 // Use this to switch between old and new accessor methods
149 #define NANOVDB_NEW_ACCESSOR_METHODS
150 
151 #define NANOVDB_FPN_BRANCHLESS
152 
153 // Do not change this value! 32 byte alignment is fixed in NanoVDB
154 #define NANOVDB_DATA_ALIGNMENT 32
155 
156 #if !defined(NANOVDB_ALIGN)
157 #define NANOVDB_ALIGN(n) alignas(n)
158 #endif // !defined(NANOVDB_ALIGN)
159 
160 #ifdef __CUDACC_RTC__
161 
162 typedef signed char int8_t;
163 typedef short int16_t;
164 typedef int int32_t;
165 typedef long long int64_t;
166 typedef unsigned char uint8_t;
167 typedef unsigned int uint32_t;
168 typedef unsigned short uint16_t;
169 typedef unsigned long long uint64_t;
170 
171 #define NANOVDB_ASSERT(x)
172 
173 #define UINT64_C(x) (x ## ULL)
174 
175 #else // !__CUDACC_RTC__
176 
177 #include <stdlib.h> // for abs in clang7
178 #include <stdint.h> // for types like int32_t etc
179 #include <stddef.h> // for size_t type
180 #include <cassert> // for assert
181 #include <cstdio> // for snprintf
182 #include <cmath> // for sqrt and fma
183 #include <limits> // for numeric_limits
184 #include <utility>// for std::move
185 #ifdef NANOVDB_USE_IOSTREAMS
186 #include <fstream>// for read/writeUncompressedGrids
187 #endif
188 // All asserts can be disabled here, even for debug builds
189 #if 1
190 #define NANOVDB_ASSERT(x) assert(x)
191 #else
192 #define NANOVDB_ASSERT(x)
193 #endif
194 
195 #if defined(NANOVDB_USE_INTRINSICS) && defined(_MSC_VER)
196 #include <intrin.h>
197 #pragma intrinsic(_BitScanReverse)
198 #pragma intrinsic(_BitScanForward)
199 #pragma intrinsic(_BitScanReverse64)
200 #pragma intrinsic(_BitScanForward64)
201 #endif
202 
203 #endif // __CUDACC_RTC__
204 
205 #if defined(__CUDACC__) || defined(__HIP__)
206 // Only define __hostdev__ when using NVIDIA CUDA or HIP compilers
207 #ifndef __hostdev__
208 #define __hostdev__ __host__ __device__ // Runs on the CPU and GPU, called from the CPU or the GPU
209 #endif
210 #else
211 // Dummy definitions of macros only defined by CUDA and HIP compilers
212 #ifndef __hostdev__
213 #define __hostdev__ // Runs on the CPU and GPU, called from the CPU or the GPU
214 #endif
215 #ifndef __global__
216 #define __global__ // Runs on the GPU, called from the CPU or the GPU
217 #endif
218 #ifndef __device__
219 #define __device__ // Runs on the GPU, called from the GPU
220 #endif
221 #ifndef __host__
222 #define __host__ // Runs on the CPU, called from the CPU
223 #endif
224 
225 #endif // if defined(__CUDACC__) || defined(__HIP__)
226 
227 // The following macro will suppress annoying warnings when nvcc
228 // compiles functions that call (host) intrinsics (which is perfectly valid)
229 #if defined(_MSC_VER) && defined(__CUDACC__)
230 #define NANOVDB_HOSTDEV_DISABLE_WARNING __pragma("hd_warning_disable")
231 #elif defined(__GNUC__) && defined(__CUDACC__)
232 #define NANOVDB_HOSTDEV_DISABLE_WARNING _Pragma("hd_warning_disable")
233 #else
234 #define NANOVDB_HOSTDEV_DISABLE_WARNING
235 #endif
236 
237 // Define compiler warnings that work with all compilers
238 //#if defined(_MSC_VER)
239 //#define NANO_WARNING(msg) _pragma("message" #msg)
240 //#else
241 //#define NANO_WARNING(msg) _Pragma("message" #msg)
242 //#endif
243 
244 // A portable implementation of offsetof - unfortunately it doesn't work with static_assert
245 #define NANOVDB_OFFSETOF(CLASS, MEMBER) ((int)(size_t)((char*)&((CLASS*)0)->MEMBER - (char*)0))
246 
247 namespace nanovdb {
248 
249 // --------------------------> Build types <------------------------------------
250 
251 /// @brief Dummy type for a voxel whose value equals an offset into an external value array
253 {
254 };
255 
256 /// @brief Dummy type for a voxel whose value equals an offset into an external value array of active values
258 {
259 };
260 
261 /// @brief Like @c ValueIndex but with a mutable mask
263 {
264 };
265 
266 /// @brief Like @c ValueOnIndex but with a mutable mask
268 {
269 };
270 
271 /// @brief Dummy type for a voxel whose value equals its binary active state
273 {
274 };
275 
276 /// @brief Dummy type for a 16 bit floating point values
277 class Half
278 {
279 };
280 
281 /// @brief Dummy type for a 4bit quantization of float point values
282 class Fp4
283 {
284 };
285 
286 /// @brief Dummy type for a 8bit quantization of float point values
287 class Fp8
288 {
289 };
290 
291 /// @brief Dummy type for a 16bit quantization of float point values
292 class Fp16
293 {
294 };
295 
296 /// @brief Dummy type for a variable bit quantization of floating point values
297 class FpN
298 {
299 };
300 
301 /// @dummy type for indexing points into voxels
302 class Point
303 {
304 };
305 //using Points = Point;// for backwards compatibility
306 
307 // --------------------------> GridType <------------------------------------
308 
309 /// @brief List of types that are currently supported by NanoVDB
310 ///
311 /// @note To expand on this list do:
312 /// 1) Add the new type between Unknown and End in the enum below
313 /// 2) Add the new type to OpenToNanoVDB::processGrid that maps OpenVDB types to GridType
314 /// 3) Verify that the ConvertTrait in NanoToOpenVDB.h works correctly with the new type
315 /// 4) Add the new type to mapToGridType (defined below) that maps NanoVDB types to GridType
316 /// 5) Add the new type to toStr (defined below)
317 enum class GridType : uint32_t { Unknown = 0, // unknown value type - should rarely be used
318  Float = 1, // single precision floating point value
319  Double = 2, // double precision floating point value
320  Int16 = 3, // half precision signed integer value
321  Int32 = 4, // single precision signed integer value
322  Int64 = 5, // double precision signed integer value
323  Vec3f = 6, // single precision floating 3D vector
324  Vec3d = 7, // double precision floating 3D vector
325  Mask = 8, // no value, just the active state
326  Half = 9, // half precision floating point value (placeholder for IEEE 754 Half)
327  UInt32 = 10, // single precision unsigned integer value
328  Boolean = 11, // boolean value, encoded in bit array
329  RGBA8 = 12, // RGBA packed into 32bit word in reverse-order, i.e. R is lowest byte.
330  Fp4 = 13, // 4bit quantization of floating point value
331  Fp8 = 14, // 8bit quantization of floating point value
332  Fp16 = 15, // 16bit quantization of floating point value
333  FpN = 16, // variable bit quantization of floating point value
334  Vec4f = 17, // single precision floating 4D vector
335  Vec4d = 18, // double precision floating 4D vector
336  Index = 19, // index into an external array of active and inactive values
337  OnIndex = 20, // index into an external array of active values
338  IndexMask = 21, // like Index but with a mutable mask
339  OnIndexMask = 22, // like OnIndex but with a mutable mask
340  PointIndex = 23, // voxels encode indices to co-located points
341  Vec3u8 = 24, // 8bit quantization of floating point 3D vector (only as blind data)
342  Vec3u16 = 25, // 16bit quantization of floating point 3D vector (only as blind data)
343  End = 26 }; // should never be used
344 
345 #ifndef __CUDACC_RTC__
346 /// @brief Maps a GridType to a c-string
347 /// @param gridType GridType to be mapped to a string
348 /// @return Retuns a c-string used to describe a GridType
349 inline const char* toStr(GridType gridType)
350 {
351  static const char* LUT[] = {"?", "float", "double", "int16", "int32", "int64", "Vec3f", "Vec3d", "Mask", "Half",
352  "uint32", "bool", "RGBA8", "Float4", "Float8", "Float16", "FloatN", "Vec4f", "Vec4d",
353  "Index", "OnIndex", "IndexMask", "OnIndexMask", "PointIndex", "Vec3u8", "Vec3u16", "End"};
354  static_assert(sizeof(LUT) / sizeof(char*) - 1 == int(GridType::End), "Unexpected size of LUT");
355  return LUT[static_cast<int>(gridType)];
356 }
357 #endif
358 
359 // --------------------------> GridClass <------------------------------------
360 
361 /// @brief Classes (superset of OpenVDB) that are currently supported by NanoVDB
362 enum class GridClass : uint32_t { Unknown = 0,
363  LevelSet = 1, // narrow band level set, e.g. SDF
364  FogVolume = 2, // fog volume, e.g. density
365  Staggered = 3, // staggered MAC grid, e.g. velocity
366  PointIndex = 4, // point index grid
367  PointData = 5, // point data grid
368  Topology = 6, // grid with active states only (no values)
369  VoxelVolume = 7, // volume of geometric cubes, e.g. colors cubes in Minecraft
370  IndexGrid = 8, // grid whose values are offsets, e.g. into an external array
371  TensorGrid = 9, // Index grid for indexing learnable tensor features
372  End = 10 };
373 
374 #ifndef __CUDACC_RTC__
375 /// @brief Retuns a c-string used to describe a GridClass
376 inline const char* toStr(GridClass gridClass)
377 {
378  static const char* LUT[] = {"?", "SDF", "FOG", "MAC", "PNTIDX", "PNTDAT", "TOPO", "VOX", "INDEX", "TENSOR", "END"};
379  static_assert(sizeof(LUT) / sizeof(char*) - 1 == int(GridClass::End), "Unexpected size of LUT");
380  return LUT[static_cast<int>(gridClass)];
381 }
382 #endif
383 
384 // --------------------------> GridFlags <------------------------------------
385 
386 /// @brief Grid flags which indicate what extra information is present in the grid buffer.
387 enum class GridFlags : uint32_t {
388  HasLongGridName = 1 << 0, // grid name is longer than 256 characters
389  HasBBox = 1 << 1, // nodes contain bounding-boxes of active values
390  HasMinMax = 1 << 2, // nodes contain min/max of active values
391  HasAverage = 1 << 3, // nodes contain averages of active values
392  HasStdDeviation = 1 << 4, // nodes contain standard deviations of active values
393  IsBreadthFirst = 1 << 5, // nodes are typically arranged breadth-first in memory
394  End = 1 << 6, // use End - 1 as a mask for the 5 lower bit flags
395 };
396 
397 #ifndef __CUDACC_RTC__
398 /// @brief Retuns a c-string used to describe a GridFlags
399 inline const char* toStr(GridFlags gridFlags)
400 {
401  static const char* LUT[] = {"has long grid name",
402  "has bbox",
403  "has min/max",
404  "has average",
405  "has standard deviation",
406  "is breadth-first",
407  "end"};
408  static_assert(1 << (sizeof(LUT) / sizeof(char*) - 1) == int(GridFlags::End), "Unexpected size of LUT");
409  return LUT[static_cast<int>(gridFlags)];
410 }
411 #endif
412 
413 // --------------------------> GridBlindData enums <------------------------------------
414 
415 /// @brief Blind-data Classes that are currently supported by NanoVDB
416 enum class GridBlindDataClass : uint32_t { Unknown = 0,
417  IndexArray = 1,
418  AttributeArray = 2,
419  GridName = 3,
420  ChannelArray = 4,
421  End = 5 };
422 
423 /// @brief Blind-data Semantics that are currently understood by NanoVDB
424 enum class GridBlindDataSemantic : uint32_t { Unknown = 0,
425  PointPosition = 1, // 3D coordinates in an unknown space
426  PointColor = 2,
427  PointNormal = 3,
428  PointRadius = 4,
429  PointVelocity = 5,
430  PointId = 6,
431  WorldCoords = 7, // 3D coordinates in world space, e.g. (0.056, 0.8, 1,8)
432  GridCoords = 8, // 3D coordinates in grid space, e.g. (1.2, 4.0, 5.7), aka index-space
433  VoxelCoords = 9, // 3D coordinates in voxel space, e.g. (0.2, 0.0, 0.7)
434  End = 10 };
435 
436 // --------------------------> is_same <------------------------------------
437 
438 /// @brief C++11 implementation of std::is_same
439 /// @note When more than two arguments are provided value = T0==T1 || T0==T2 || ...
440 template<typename T0, typename T1, typename ...T>
441 struct is_same
442 {
443  static constexpr bool value = is_same<T0, T1>::value || is_same<T0, T...>::value;
444 };
445 
446 template<typename T0, typename T1>
447 struct is_same<T0, T1>
448 {
449  static constexpr bool value = false;
450 };
451 
452 template<typename T>
453 struct is_same<T, T>
454 {
455  static constexpr bool value = true;
456 };
457 
458 // --------------------------> is_floating_point <------------------------------------
459 
460 /// @brief C++11 implementation of std::is_floating_point
461 template<typename T>
463 {
464  static constexpr bool value = is_same<T, float, double>::value;
465 };
466 
467 // --------------------------> BuildTraits <------------------------------------
468 
469 /// @brief Define static boolean tests for template build types
470 template<typename T>
472 {
473  // check if T is an index type
475  static constexpr bool is_onindex = is_same<T, ValueOnIndex, ValueOnIndexMask>::value;
476  static constexpr bool is_offindex = is_same<T, ValueIndex, ValueIndexMask>::value;
477  static constexpr bool is_indexmask = is_same<T, ValueIndexMask, ValueOnIndexMask>::value;
478  // check if T is a compressed float type with fixed bit precision
479  static constexpr bool is_FpX = is_same<T, Fp4, Fp8, Fp16>::value;
480  // check if T is a compressed float type with fixed or variable bit precision
481  static constexpr bool is_Fp = is_same<T, Fp4, Fp8, Fp16, FpN>::value;
482  // check if T is a POD float type, i.e float or double
483  static constexpr bool is_float = is_floating_point<T>::value;
484  // check if T is a template specialization of LeafData<T>, i.e. has T mValues[512]
485  static constexpr bool is_special = is_index || is_Fp || is_same<T, Point, bool, ValueMask>::value;
486 }; // BuildTraits
487 
488 // --------------------------> enable_if <------------------------------------
489 
490 /// @brief C++11 implementation of std::enable_if
491 template <bool, typename T = void>
492 struct enable_if
493 {
494 };
495 
496 template <typename T>
497 struct enable_if<true, T>
498 {
499  using type = T;
500 };
501 
502 // --------------------------> disable_if <------------------------------------
503 
504 template<bool, typename T = void>
506 {
507  typedef T type;
508 };
509 
510 template<typename T>
511 struct disable_if<true, T>
512 {
513 };
514 
515 // --------------------------> is_const <------------------------------------
516 
517 template<typename T>
518 struct is_const
519 {
520  static constexpr bool value = false;
521 };
522 
523 template<typename T>
524 struct is_const<const T>
525 {
526  static constexpr bool value = true;
527 };
528 
529 // --------------------------> is_pointer <------------------------------------
530 
531 /// @brief Trait used to identify template parameter that are pointers
532 /// @tparam T Template parameter to be tested
533 template<class T>
535 {
536  static constexpr bool value = false;
537 };
538 
539 /// @brief Template specialization of non-const pointers
540 /// @tparam T Template parameter to be tested
541 template<class T>
542 struct is_pointer<T*>
543 {
544  static constexpr bool value = true;
545 };
546 
547 /// @brief Template specialization of const pointers
548 /// @tparam T Template parameter to be tested
549 template<class T>
550 struct is_pointer<const T*>
551 {
552  static constexpr bool value = true;
553 };
554 
555 // --------------------------> remove_const <------------------------------------
556 
557 /// @brief Trait use to const from type. Default implementation is just a pass-through
558 /// @tparam T Type
559 /// @details remove_pointer<float>::type = float
560 template<typename T>
562 {
563  using type = T;
564 };
565 
566 /// @brief Template specialization of trait class use to remove const qualifier type from a type
567 /// @tparam T Type of the const type
568 /// @details remove_pointer<const float>::type = float
569 template<typename T>
570 struct remove_const<const T>
571 {
572  using type = T;
573 };
574 
575 // --------------------------> remove_reference <------------------------------------
576 
577 /// @brief Trait use to remove reference, i.e. "&", qualifier from a type. Default implementation is just a pass-through
578 /// @tparam T Type
579 /// @details remove_pointer<float>::type = float
580 template <typename T>
581 struct remove_reference {using type = T;};
582 
583 /// @brief Template specialization of trait class use to remove reference, i.e. "&", qualifier from a type
584 /// @tparam T Type of the reference
585 /// @details remove_pointer<float&>::type = float
586 template <typename T>
587 struct remove_reference<T&> {using type = T;};
588 
589 // --------------------------> remove_pointer <------------------------------------
590 
591 /// @brief Trait use to remove pointer, i.e. "*", qualifier from a type. Default implementation is just a pass-through
592 /// @tparam T Type
593 /// @details remove_pointer<float>::type = float
594 template <typename T>
595 struct remove_pointer {using type = T;};
596 
597 /// @brief Template specialization of trait class use to to remove pointer, i.e. "*", qualifier from a type
598 /// @tparam T Type of the pointer
599 /// @details remove_pointer<float*>::type = float
600 template <typename T>
601 struct remove_pointer<T*> {using type = T;};
602 
603 // --------------------------> match_const <------------------------------------
604 
605 /// @brief Trait used to transfer the const-ness of a reference type to another type
606 /// @tparam T Type whose const-ness needs to match the reference type
607 /// @tparam ReferenceT Reference type that is not const
608 /// @details match_const<const int, float>::type = int
609 /// match_const<int, float>::type = int
610 template<typename T, typename ReferenceT>
612 {
613  using type = typename remove_const<T>::type;
614 };
615 
616 /// @brief Template specialization used to transfer the const-ness of a reference type to another type
617 /// @tparam T Type that will adopt the const-ness of the reference type
618 /// @tparam ReferenceT Reference type that is const
619 /// @details match_const<const int, const float>::type = const int
620 /// match_const<int, const float>::type = const int
621 template<typename T, typename ReferenceT>
622 struct match_const<T, const ReferenceT>
623 {
624  using type = const typename remove_const<T>::type;
625 };
626 
627 // --------------------------> is_specialization <------------------------------------
628 
629 /// @brief Metafunction used to determine if the first template
630 /// parameter is a specialization of the class template
631 /// given in the second template parameter.
632 ///
633 /// @details is_specialization<Vec3<float>, Vec3>::value == true;
634 /// is_specialization<Vec3f, Vec3>::value == true;
635 /// is_specialization<std::vector<float>, std::vector>::value == true;
636 template<typename AnyType, template<typename...> class TemplateType>
638 {
639  static const bool value = false;
640 };
641 template<typename... Args, template<typename...> class TemplateType>
642 struct is_specialization<TemplateType<Args...>, TemplateType>
643 {
644  static const bool value = true;
645 };
646 
647 // --------------------------> BuildToValueMap <------------------------------------
648 
649 /// @brief Maps one type (e.g. the build types above) to other (actual) types
650 template<typename T>
652 {
653  using Type = T;
654  using type = T;
655 };
656 
657 template<>
659 {
660  using Type = uint64_t;
661  using type = uint64_t;
662 };
663 
664 template<>
666 {
667  using Type = uint64_t;
668  using type = uint64_t;
669 };
670 
671 template<>
673 {
674  using Type = uint64_t;
675  using type = uint64_t;
676 };
677 
678 template<>
680 {
681  using Type = uint64_t;
682  using type = uint64_t;
683 };
684 
685 template<>
687 {
688  using Type = bool;
689  using type = bool;
690 };
691 
692 template<>
694 {
695  using Type = float;
696  using type = float;
697 };
698 
699 template<>
701 {
702  using Type = float;
703  using type = float;
704 };
705 
706 template<>
708 {
709  using Type = float;
710  using type = float;
711 };
712 
713 template<>
715 {
716  using Type = float;
717  using type = float;
718 };
719 
720 template<>
722 {
723  using Type = float;
724  using type = float;
725 };
726 
727 template<>
729 {
730  using Type = uint64_t;
731  using type = uint64_t;
732 };
733 
734 // --------------------------> utility functions related to alignment <------------------------------------
735 
736 /// @brief return true if the specified pointer is aligned
737 __hostdev__ inline static bool isAligned(const void* p)
738 {
739  return uint64_t(p) % NANOVDB_DATA_ALIGNMENT == 0;
740 }
741 
742 /// @brief return true if the specified pointer is aligned and not NULL
743 __hostdev__ inline static bool isValid(const void* p)
744 {
745  return p != nullptr && uint64_t(p) % NANOVDB_DATA_ALIGNMENT == 0;
746 }
747 
748 /// @brief return the smallest number of bytes that when added to the specified pointer results in an aligned pointer
749 __hostdev__ inline static uint64_t alignmentPadding(const void* p)
750 {
751  NANOVDB_ASSERT(p);
753 }
754 
755 /// @brief offset the specified pointer so it is aligned.
756 template <typename T>
757 __hostdev__ inline static T* alignPtr(T* p)
758 {
759  NANOVDB_ASSERT(p);
760  return reinterpret_cast<T*>( (uint8_t*)p + alignmentPadding(p) );
761 }
762 
763 /// @brief offset the specified pointer so it is aligned.
764 template <typename T>
765 __hostdev__ inline static const T* alignPtr(const T* p)
766 {
767  NANOVDB_ASSERT(p);
768  return reinterpret_cast<const T*>( (const uint8_t*)p + alignmentPadding(p) );
769 }
770 
771 // --------------------------> PtrDiff <------------------------------------
772 
773 /// @brief Compute the distance, in bytes, between two pointers
774 /// @tparam T1 Type of the first pointer
775 /// @tparam T2 Type of the second pointer
776 /// @param p fist pointer, assumed to NOT be NULL
777 /// @param q second pointer, assumed to NOT be NULL
778 /// @return signed distance between pointer addresses in units of bytes
779 template<typename T1, typename T2>
780 __hostdev__ inline static int64_t PtrDiff(const T1* p, const T2* q)
781 {
782  NANOVDB_ASSERT(p && q);
783  return reinterpret_cast<const char*>(p) - reinterpret_cast<const char*>(q);
784 }
785 
786 // --------------------------> PtrAdd <------------------------------------
787 
788 /// @brief Adds a byte offset of a non-const pointer to produce another non-const pointer
789 /// @tparam DstT Type of the return pointer
790 /// @tparam SrcT Type of the input pointer
791 /// @param p non-const input pointer, assumed to NOT be NULL
792 /// @param offset signed byte offset
793 /// @return a non-const pointer defined as the offset of an input pointer
794 template<typename DstT, typename SrcT>
795 __hostdev__ inline static DstT* PtrAdd(SrcT* p, int64_t offset)
796 {
797  NANOVDB_ASSERT(p);
798  return reinterpret_cast<DstT*>(reinterpret_cast<char*>(p) + offset);
799 }
800 
801 /// @brief Adds a byte offset of a const pointer to produce another const pointer
802 /// @tparam DstT Type of the return pointer
803 /// @tparam SrcT Type of the input pointer
804 /// @param p const input pointer, assumed to NOT be NULL
805 /// @param offset signed byte offset
806 /// @return a const pointer defined as the offset of a const input pointer
807 template<typename DstT, typename SrcT>
808 __hostdev__ inline static const DstT* PtrAdd(const SrcT* p, int64_t offset)
809 {
810  NANOVDB_ASSERT(p);
811  return reinterpret_cast<const DstT*>(reinterpret_cast<const char*>(p) + offset);
812 }
813 
814 // --------------------------> isFloatingPoint(GridType) <------------------------------------
815 
816 /// @brief return true if the GridType maps to a floating point type
817 __hostdev__ inline bool isFloatingPoint(GridType gridType)
818 {
819  return gridType == GridType::Float ||
820  gridType == GridType::Double ||
821  gridType == GridType::Half ||
822  gridType == GridType::Fp4 ||
823  gridType == GridType::Fp8 ||
824  gridType == GridType::Fp16 ||
825  gridType == GridType::FpN;
826 }
827 
828 // --------------------------> isFloatingPointVector(GridType) <------------------------------------
829 
830 /// @brief return true if the GridType maps to a floating point vec3.
832 {
833  return gridType == GridType::Vec3f ||
834  gridType == GridType::Vec3d ||
835  gridType == GridType::Vec4f ||
836  gridType == GridType::Vec4d;
837 }
838 
839 // --------------------------> isInteger(GridType) <------------------------------------
840 
841 /// @brief Return true if the GridType maps to a POD integer type.
842 /// @details These types are used to associate a voxel with a POD integer type
843 __hostdev__ inline bool isInteger(GridType gridType)
844 {
845  return gridType == GridType::Int16 ||
846  gridType == GridType::Int32 ||
847  gridType == GridType::Int64 ||
848  gridType == GridType::UInt32;
849 }
850 
851 // --------------------------> isIndex(GridType) <------------------------------------
852 
853 /// @brief Return true if the GridType maps to a special index type (not a POD integer type).
854 /// @details These types are used to index from a voxel into an external array of values, e.g. sidecar or blind data.
855 __hostdev__ inline bool isIndex(GridType gridType)
856 {
857  return gridType == GridType::Index ||// index both active and inactive values
858  gridType == GridType::OnIndex ||// index active values only
859  gridType == GridType::IndexMask ||// as Index, but with an additional mask
860  gridType == GridType::OnIndexMask;// as OnIndex, but with an additional mask
861 }
862 
863 // --------------------------> memcpy64 <------------------------------------
864 
865 /// @brief copy 64 bit words from @c src to @c dst
866 /// @param dst pointer to destination
867 /// @param src pointer to source
868 /// @param word_count number of 64 bit words to be copied
869 /// @return destination pointer
870 /// @warning @c src and @c dst cannot overlap and should both be 64 bit aligned
871 __hostdev__ inline static void* memcpy64(void *dst, const void *src, size_t word_count)
872 {
873  NANOVDB_ASSERT(uint64_t(dst) % 8 == 0 && uint64_t(src) % 8 == 0);
874  auto *d = reinterpret_cast<uint64_t*>(dst), *e = d + word_count;
875  auto *s = reinterpret_cast<const uint64_t*>(src);
876  while (d != e) *d++ = *s++;
877  return dst;
878 }
879 
880 // --------------------------> isValue(GridType, GridClass) <------------------------------------
881 
882 /// @brief return true if the combination of GridType and GridClass is valid.
883 __hostdev__ inline bool isValid(GridType gridType, GridClass gridClass)
884 {
885  if (gridClass == GridClass::LevelSet || gridClass == GridClass::FogVolume) {
886  return isFloatingPoint(gridType);
887  } else if (gridClass == GridClass::Staggered) {
888  return isFloatingPointVector(gridType);
889  } else if (gridClass == GridClass::PointIndex || gridClass == GridClass::PointData) {
890  return gridType == GridType::PointIndex || gridType == GridType::UInt32;
891  } else if (gridClass == GridClass::Topology) {
892  return gridType == GridType::Mask;
893  } else if (gridClass == GridClass::IndexGrid) {
894  return isIndex(gridType);
895  } else if (gridClass == GridClass::VoxelVolume) {
896  return gridType == GridType::RGBA8 || gridType == GridType::Float ||
897  gridType == GridType::Double || gridType == GridType::Vec3f ||
898  gridType == GridType::Vec3d || gridType == GridType::UInt32;
899  }
900  return gridClass < GridClass::End && gridType < GridType::End; // any valid combination
901 }
902 
903 // --------------------------> validation of blind data meta data <------------------------------------
904 
905 /// @brief return true if the combination of GridBlindDataClass, GridBlindDataSemantic and GridType is valid.
906 __hostdev__ inline bool isValid(const GridBlindDataClass& blindClass,
907  const GridBlindDataSemantic& blindSemantics,
908  const GridType& blindType)
909 {
910  bool test = false;
911  switch (blindClass) {
913  test = (blindSemantics == GridBlindDataSemantic::Unknown ||
914  blindSemantics == GridBlindDataSemantic::PointId) &&
915  isInteger(blindType);
916  break;
918  if (blindSemantics == GridBlindDataSemantic::PointPosition ||
919  blindSemantics == GridBlindDataSemantic::WorldCoords) {
920  test = blindType == GridType::Vec3f || blindType == GridType::Vec3d;
921  } else if (blindSemantics == GridBlindDataSemantic::GridCoords) {
922  test = blindType == GridType::Vec3f;
923  } else if (blindSemantics == GridBlindDataSemantic::VoxelCoords) {
924  test = blindType == GridType::Vec3f || blindType == GridType::Vec3u8 || blindType == GridType::Vec3u16;
925  } else {
926  test = blindSemantics != GridBlindDataSemantic::PointId;
927  }
928  break;
930  test = blindSemantics == GridBlindDataSemantic::Unknown && blindType == GridType::Unknown;
931  break;
932  default: // captures blindClass == Unknown and ChannelArray
933  test = blindClass < GridBlindDataClass::End &&
934  blindSemantics < GridBlindDataSemantic::End &&
935  blindType < GridType::End; // any valid combination
936  break;
937  }
938  //if (!test) printf("Invalid combination: GridBlindDataClass=%u, GridBlindDataSemantic=%u, GridType=%u\n",(uint32_t)blindClass, (uint32_t)blindSemantics, (uint32_t)blindType);
939  return test;
940 }
941 
942 // ----------------------------> Version class <-------------------------------------
943 
944 /// @brief Bit-compacted representation of all three version numbers
945 ///
946 /// @details major is the top 11 bits, minor is the 11 middle bits and patch is the lower 10 bits
947 class Version
948 {
949  uint32_t mData; // 11 + 11 + 10 bit packing of major + minor + patch
950 public:
952  : mData(uint32_t(NANOVDB_MAJOR_VERSION_NUMBER) << 21 |
953  uint32_t(NANOVDB_MINOR_VERSION_NUMBER) << 10 |
955  {
956  }
957  __hostdev__ Version(uint32_t data) : mData(data) {}
958  __hostdev__ Version(uint32_t major, uint32_t minor, uint32_t patch)
959  : mData(major << 21 | minor << 10 | patch)
960  {
961  NANOVDB_ASSERT(major < (1u << 11)); // max value of major is 2047
962  NANOVDB_ASSERT(minor < (1u << 11)); // max value of minor is 2047
963  NANOVDB_ASSERT(patch < (1u << 10)); // max value of patch is 1023
964  }
965  __hostdev__ bool operator==(const Version& rhs) const { return mData == rhs.mData; }
966  __hostdev__ bool operator<( const Version& rhs) const { return mData < rhs.mData; }
967  __hostdev__ bool operator<=(const Version& rhs) const { return mData <= rhs.mData; }
968  __hostdev__ bool operator>( const Version& rhs) const { return mData > rhs.mData; }
969  __hostdev__ bool operator>=(const Version& rhs) const { return mData >= rhs.mData; }
970  __hostdev__ uint32_t id() const { return mData; }
971  __hostdev__ uint32_t getMajor() const { return (mData >> 21) & ((1u << 11) - 1); }
972  __hostdev__ uint32_t getMinor() const { return (mData >> 10) & ((1u << 11) - 1); }
973  __hostdev__ uint32_t getPatch() const { return mData & ((1u << 10) - 1); }
974  __hostdev__ bool isCompatible() const { return this->getMajor() == uint32_t(NANOVDB_MAJOR_VERSION_NUMBER);}
975  /// @brief Check the major version of this instance relative to NANOVDB_MAJOR_VERSION_NUMBER
976  /// @return return 0 if the major version equals NANOVDB_MAJOR_VERSION_NUMBER, else a negative age if it is
977  /// older, i.e. smaller, and a positive age if it's newer, i.e.e larger.
978  __hostdev__ int age() const {return int(this->getMajor()) - int(NANOVDB_MAJOR_VERSION_NUMBER);}
979 
980 #ifndef __CUDACC_RTC__
981  const char* c_str() const
982  {
983  char* buffer = (char*)malloc(4 + 1 + 4 + 1 + 4 + 1); // xxxx.xxxx.xxxx\0
984  snprintf(buffer, 4 + 1 + 4 + 1 + 4 + 1, "%u.%u.%u", this->getMajor(), this->getMinor(), this->getPatch()); // Prevents overflows by enforcing a fixed size of buffer
985  return buffer;
986  }
987 #endif
988 }; // Version
989 
990 // ----------------------------> Various math functions <-------------------------------------
991 
992 //@{
993 /// @brief Pi constant taken from Boost to match old behaviour
994 template<typename T>
995 inline __hostdev__ constexpr T pi()
996 {
997  return 3.141592653589793238462643383279502884e+00;
998 }
999 template<>
1000 inline __hostdev__ constexpr float pi()
1001 {
1002  return 3.141592653589793238462643383279502884e+00F;
1003 }
1004 template<>
1005 inline __hostdev__ constexpr double pi()
1006 {
1007  return 3.141592653589793238462643383279502884e+00;
1008 }
1009 template<>
1010 inline __hostdev__ constexpr long double pi()
1011 {
1012  return 3.141592653589793238462643383279502884e+00L;
1013 }
1014 //@}
1015 
1016 //@{
1017 /// Tolerance for floating-point comparison
1018 template<typename T>
1019 struct Tolerance;
1020 template<>
1021 struct Tolerance<float>
1022 {
1023  __hostdev__ static float value() { return 1e-8f; }
1024 };
1025 template<>
1026 struct Tolerance<double>
1027 {
1028  __hostdev__ static double value() { return 1e-15; }
1029 };
1030 //@}
1031 
1032 //@{
1033 /// Delta for small floating-point offsets
1034 template<typename T>
1035 struct Delta;
1036 template<>
1037 struct Delta<float>
1038 {
1039  __hostdev__ static float value() { return 1e-5f; }
1040 };
1041 template<>
1042 struct Delta<double>
1043 {
1044  __hostdev__ static double value() { return 1e-9; }
1045 };
1046 //@}
1047 
1048 //@{
1049 /// Maximum floating-point values
1050 template<typename T>
1051 struct Maximum;
1052 #if defined(__CUDA_ARCH__) || defined(__HIP__)
1053 template<>
1054 struct Maximum<int>
1055 {
1056  __hostdev__ static int value() { return 2147483647; }
1057 };
1058 template<>
1059 struct Maximum<uint32_t>
1060 {
1061  __hostdev__ static uint32_t value() { return 4294967295u; }
1062 };
1063 template<>
1064 struct Maximum<float>
1065 {
1066  __hostdev__ static float value() { return 1e+38f; }
1067 };
1068 template<>
1069 struct Maximum<double>
1070 {
1071  __hostdev__ static double value() { return 1e+308; }
1072 };
1073 #else
1074 template<typename T>
1075 struct Maximum
1076 {
1077  static T value() { return std::numeric_limits<T>::max(); }
1078 };
1079 #endif
1080 //@}
1081 
1082 template<typename Type>
1083 __hostdev__ inline bool isApproxZero(const Type& x)
1084 {
1085  return !(x > Tolerance<Type>::value()) && !(x < -Tolerance<Type>::value());
1086 }
1087 
1088 template<typename Type>
1089 __hostdev__ inline Type Min(Type a, Type b)
1090 {
1091  return (a < b) ? a : b;
1092 }
1093 __hostdev__ inline int32_t Min(int32_t a, int32_t b)
1094 {
1095  return int32_t(fminf(float(a), float(b)));
1096 }
1097 __hostdev__ inline uint32_t Min(uint32_t a, uint32_t b)
1098 {
1099  return uint32_t(fminf(float(a), float(b)));
1100 }
1101 __hostdev__ inline float Min(float a, float b)
1102 {
1103  return fminf(a, b);
1104 }
1105 __hostdev__ inline double Min(double a, double b)
1106 {
1107  return fmin(a, b);
1108 }
1109 template<typename Type>
1110 __hostdev__ inline Type Max(Type a, Type b)
1111 {
1112  return (a > b) ? a : b;
1113 }
1114 
1115 __hostdev__ inline int32_t Max(int32_t a, int32_t b)
1116 {
1117  return int32_t(fmaxf(float(a), float(b)));
1118 }
1119 __hostdev__ inline uint32_t Max(uint32_t a, uint32_t b)
1120 {
1121  return uint32_t(fmaxf(float(a), float(b)));
1122 }
1123 __hostdev__ inline float Max(float a, float b)
1124 {
1125  return fmaxf(a, b);
1126 }
1127 __hostdev__ inline double Max(double a, double b)
1128 {
1129  return fmax(a, b);
1130 }
1131 __hostdev__ inline float Clamp(float x, float a, float b)
1132 {
1133  return Max(Min(x, b), a);
1134 }
1135 __hostdev__ inline double Clamp(double x, double a, double b)
1136 {
1137  return Max(Min(x, b), a);
1138 }
1139 
1140 __hostdev__ inline float Fract(float x)
1141 {
1142  return x - floorf(x);
1143 }
1144 __hostdev__ inline double Fract(double x)
1145 {
1146  return x - floor(x);
1147 }
1148 
1149 __hostdev__ inline int32_t Floor(float x)
1150 {
1151  return int32_t(floorf(x));
1152 }
1153 __hostdev__ inline int32_t Floor(double x)
1154 {
1155  return int32_t(floor(x));
1156 }
1157 
1158 __hostdev__ inline int32_t Ceil(float x)
1159 {
1160  return int32_t(ceilf(x));
1161 }
1162 __hostdev__ inline int32_t Ceil(double x)
1163 {
1164  return int32_t(ceil(x));
1165 }
1166 
1167 template<typename T>
1168 __hostdev__ inline T Pow2(T x)
1169 {
1170  return x * x;
1171 }
1172 
1173 template<typename T>
1174 __hostdev__ inline T Pow3(T x)
1175 {
1176  return x * x * x;
1177 }
1178 
1179 template<typename T>
1180 __hostdev__ inline T Pow4(T x)
1181 {
1182  return Pow2(x * x);
1183 }
1184 template<typename T>
1185 __hostdev__ inline T Abs(T x)
1186 {
1187  return x < 0 ? -x : x;
1188 }
1189 
1190 template<>
1191 __hostdev__ inline float Abs(float x)
1192 {
1193  return fabsf(x);
1194 }
1195 
1196 template<>
1197 __hostdev__ inline double Abs(double x)
1198 {
1199  return fabs(x);
1200 }
1201 
1202 template<>
1203 __hostdev__ inline int Abs(int x)
1204 {
1205  return abs(x);
1206 }
1207 
1208 template<typename CoordT, typename RealT, template<typename> class Vec3T>
1209 __hostdev__ inline CoordT Round(const Vec3T<RealT>& xyz);
1210 
1211 template<typename CoordT, template<typename> class Vec3T>
1212 __hostdev__ inline CoordT Round(const Vec3T<float>& xyz)
1213 {
1214  return CoordT(int32_t(rintf(xyz[0])), int32_t(rintf(xyz[1])), int32_t(rintf(xyz[2])));
1215  //return CoordT(int32_t(roundf(xyz[0])), int32_t(roundf(xyz[1])), int32_t(roundf(xyz[2])) );
1216  //return CoordT(int32_t(floorf(xyz[0] + 0.5f)), int32_t(floorf(xyz[1] + 0.5f)), int32_t(floorf(xyz[2] + 0.5f)));
1217 }
1218 
1219 template<typename CoordT, template<typename> class Vec3T>
1220 __hostdev__ inline CoordT Round(const Vec3T<double>& xyz)
1221 {
1222  return CoordT(int32_t(floor(xyz[0] + 0.5)), int32_t(floor(xyz[1] + 0.5)), int32_t(floor(xyz[2] + 0.5)));
1223 }
1224 
1225 template<typename CoordT, typename RealT, template<typename> class Vec3T>
1226 __hostdev__ inline CoordT RoundDown(const Vec3T<RealT>& xyz)
1227 {
1228  return CoordT(Floor(xyz[0]), Floor(xyz[1]), Floor(xyz[2]));
1229 }
1230 
1231 //@{
1232 /// Return the square root of a floating-point value.
1233 __hostdev__ inline float Sqrt(float x)
1234 {
1235  return sqrtf(x);
1236 }
1237 __hostdev__ inline double Sqrt(double x)
1238 {
1239  return sqrt(x);
1240 }
1241 //@}
1242 
1243 /// Return the sign of the given value as an integer (either -1, 0 or 1).
1244 template<typename T>
1245 __hostdev__ inline T Sign(const T& x)
1246 {
1247  return ((T(0) < x) ? T(1) : T(0)) - ((x < T(0)) ? T(1) : T(0));
1248 }
1249 
1250 template<typename Vec3T>
1251 __hostdev__ inline int MinIndex(const Vec3T& v)
1252 {
1253 #if 0
1254  static const int hashTable[8] = {2, 1, 9, 1, 2, 9, 0, 0}; //9 are dummy values
1255  const int hashKey = ((v[0] < v[1]) << 2) + ((v[0] < v[2]) << 1) + (v[1] < v[2]); // ?*4+?*2+?*1
1256  return hashTable[hashKey];
1257 #else
1258  if (v[0] < v[1] && v[0] < v[2])
1259  return 0;
1260  if (v[1] < v[2])
1261  return 1;
1262  else
1263  return 2;
1264 #endif
1265 }
1266 
1267 template<typename Vec3T>
1268 __hostdev__ inline int MaxIndex(const Vec3T& v)
1269 {
1270 #if 0
1271  static const int hashTable[8] = {2, 1, 9, 1, 2, 9, 0, 0}; //9 are dummy values
1272  const int hashKey = ((v[0] > v[1]) << 2) + ((v[0] > v[2]) << 1) + (v[1] > v[2]); // ?*4+?*2+?*1
1273  return hashTable[hashKey];
1274 #else
1275  if (v[0] > v[1] && v[0] > v[2])
1276  return 0;
1277  if (v[1] > v[2])
1278  return 1;
1279  else
1280  return 2;
1281 #endif
1282 }
1283 
1284 /// @brief round up byteSize to the nearest wordSize, e.g. to align to machine word: AlignUp<sizeof(size_t)(n)
1285 ///
1286 /// @details both wordSize and byteSize are in byte units
1287 template<uint64_t wordSize>
1288 __hostdev__ inline uint64_t AlignUp(uint64_t byteCount)
1289 {
1290  const uint64_t r = byteCount % wordSize;
1291  return r ? byteCount - r + wordSize : byteCount;
1292 }
1293 
1294 // ------------------------------> Coord <--------------------------------------
1295 
1296 // forward declaration so we can define Coord::asVec3s and Coord::asVec3d
1297 template<typename>
1298 class Vec3;
1299 
1300 /// @brief Signed (i, j, k) 32-bit integer coordinate class, similar to openvdb::math::Coord
1301 class Coord
1302 {
1303  int32_t mVec[3]; // private member data - three signed index coordinates
1304 public:
1305  using ValueType = int32_t;
1306  using IndexType = uint32_t;
1307 
1308  /// @brief Initialize all coordinates to zero.
1310  : mVec{0, 0, 0}
1311  {
1312  }
1313 
1314  /// @brief Initializes all coordinates to the given signed integer.
1316  : mVec{n, n, n}
1317  {
1318  }
1319 
1320  /// @brief Initializes coordinate to the given signed integers.
1322  : mVec{i, j, k}
1323  {
1324  }
1325 
1327  : mVec{ptr[0], ptr[1], ptr[2]}
1328  {
1329  }
1330 
1331  __hostdev__ int32_t x() const { return mVec[0]; }
1332  __hostdev__ int32_t y() const { return mVec[1]; }
1333  __hostdev__ int32_t z() const { return mVec[2]; }
1334 
1335  __hostdev__ int32_t& x() { return mVec[0]; }
1336  __hostdev__ int32_t& y() { return mVec[1]; }
1337  __hostdev__ int32_t& z() { return mVec[2]; }
1338 
1339  __hostdev__ static Coord max() { return Coord(int32_t((1u << 31) - 1)); }
1340 
1341  __hostdev__ static Coord min() { return Coord(-int32_t((1u << 31) - 1) - 1); }
1342 
1343  __hostdev__ static size_t memUsage() { return sizeof(Coord); }
1344 
1345  /// @brief Return a const reference to the given Coord component.
1346  /// @warning The argument is assumed to be 0, 1, or 2.
1347  __hostdev__ const ValueType& operator[](IndexType i) const { return mVec[i]; }
1348 
1349  /// @brief Return a non-const reference to the given Coord component.
1350  /// @warning The argument is assumed to be 0, 1, or 2.
1351  __hostdev__ ValueType& operator[](IndexType i) { return mVec[i]; }
1352 
1353  /// @brief Assignment operator that works with openvdb::Coord
1354  template<typename CoordT>
1355  __hostdev__ Coord& operator=(const CoordT& other)
1356  {
1357  static_assert(sizeof(Coord) == sizeof(CoordT), "Mis-matched sizeof");
1358  mVec[0] = other[0];
1359  mVec[1] = other[1];
1360  mVec[2] = other[2];
1361  return *this;
1362  }
1363 
1364  /// @brief Return a new instance with coordinates masked by the given unsigned integer.
1365  __hostdev__ Coord operator&(IndexType n) const { return Coord(mVec[0] & n, mVec[1] & n, mVec[2] & n); }
1366 
1367  // @brief Return a new instance with coordinates left-shifted by the given unsigned integer.
1368  __hostdev__ Coord operator<<(IndexType n) const { return Coord(mVec[0] << n, mVec[1] << n, mVec[2] << n); }
1369 
1370  // @brief Return a new instance with coordinates right-shifted by the given unsigned integer.
1371  __hostdev__ Coord operator>>(IndexType n) const { return Coord(mVec[0] >> n, mVec[1] >> n, mVec[2] >> n); }
1372 
1373  /// @brief Return true if this Coord is lexicographically less than the given Coord.
1374  __hostdev__ bool operator<(const Coord& rhs) const
1375  {
1376  return mVec[0] < rhs[0] ? true
1377  : mVec[0] > rhs[0] ? false
1378  : mVec[1] < rhs[1] ? true
1379  : mVec[1] > rhs[1] ? false
1380  : mVec[2] < rhs[2] ? true : false;
1381  }
1382 
1383  /// @brief Return true if this Coord is lexicographically less or equal to the given Coord.
1384  __hostdev__ bool operator<=(const Coord& rhs) const
1385  {
1386  return mVec[0] < rhs[0] ? true
1387  : mVec[0] > rhs[0] ? false
1388  : mVec[1] < rhs[1] ? true
1389  : mVec[1] > rhs[1] ? false
1390  : mVec[2] <=rhs[2] ? true : false;
1391  }
1392 
1393  // @brief Return true if the Coord components are identical.
1394  __hostdev__ bool operator==(const Coord& rhs) const { return mVec[0] == rhs[0] && mVec[1] == rhs[1] && mVec[2] == rhs[2]; }
1395  __hostdev__ bool operator!=(const Coord& rhs) const { return mVec[0] != rhs[0] || mVec[1] != rhs[1] || mVec[2] != rhs[2]; }
1397  {
1398  mVec[0] &= n;
1399  mVec[1] &= n;
1400  mVec[2] &= n;
1401  return *this;
1402  }
1404  {
1405  mVec[0] <<= n;
1406  mVec[1] <<= n;
1407  mVec[2] <<= n;
1408  return *this;
1409  }
1411  {
1412  mVec[0] >>= n;
1413  mVec[1] >>= n;
1414  mVec[2] >>= n;
1415  return *this;
1416  }
1418  {
1419  mVec[0] += n;
1420  mVec[1] += n;
1421  mVec[2] += n;
1422  return *this;
1423  }
1424  __hostdev__ Coord operator+(const Coord& rhs) const { return Coord(mVec[0] + rhs[0], mVec[1] + rhs[1], mVec[2] + rhs[2]); }
1425  __hostdev__ Coord operator-(const Coord& rhs) const { return Coord(mVec[0] - rhs[0], mVec[1] - rhs[1], mVec[2] - rhs[2]); }
1426  __hostdev__ Coord operator-() const { return Coord(-mVec[0], -mVec[1], -mVec[2]); }
1428  {
1429  mVec[0] += rhs[0];
1430  mVec[1] += rhs[1];
1431  mVec[2] += rhs[2];
1432  return *this;
1433  }
1435  {
1436  mVec[0] -= rhs[0];
1437  mVec[1] -= rhs[1];
1438  mVec[2] -= rhs[2];
1439  return *this;
1440  }
1441 
1442  /// @brief Perform a component-wise minimum with the other Coord.
1444  {
1445  if (other[0] < mVec[0])
1446  mVec[0] = other[0];
1447  if (other[1] < mVec[1])
1448  mVec[1] = other[1];
1449  if (other[2] < mVec[2])
1450  mVec[2] = other[2];
1451  return *this;
1452  }
1453 
1454  /// @brief Perform a component-wise maximum with the other Coord.
1456  {
1457  if (other[0] > mVec[0])
1458  mVec[0] = other[0];
1459  if (other[1] > mVec[1])
1460  mVec[1] = other[1];
1461  if (other[2] > mVec[2])
1462  mVec[2] = other[2];
1463  return *this;
1464  }
1465 #if defined(__CUDACC__) // the following functions only run on the GPU!
1466  __device__ inline Coord& minComponentAtomic(const Coord& other)
1467  {
1468  atomicMin(&mVec[0], other[0]);
1469  atomicMin(&mVec[1], other[1]);
1470  atomicMin(&mVec[2], other[2]);
1471  return *this;
1472  }
1473  __device__ inline Coord& maxComponentAtomic(const Coord& other)
1474  {
1475  atomicMax(&mVec[0], other[0]);
1476  atomicMax(&mVec[1], other[1]);
1477  atomicMax(&mVec[2], other[2]);
1478  return *this;
1479  }
1480 #endif
1481 
1483  {
1484  return Coord(mVec[0] + dx, mVec[1] + dy, mVec[2] + dz);
1485  }
1486 
1487  __hostdev__ Coord offsetBy(ValueType n) const { return this->offsetBy(n, n, n); }
1488 
1489  /// Return true if any of the components of @a a are smaller than the
1490  /// corresponding components of @a b.
1491  __hostdev__ static inline bool lessThan(const Coord& a, const Coord& b)
1492  {
1493  return (a[0] < b[0] || a[1] < b[1] || a[2] < b[2]);
1494  }
1495 
1496  /// @brief Return the largest integer coordinates that are not greater
1497  /// than @a xyz (node centered conversion).
1498  template<typename Vec3T>
1499  __hostdev__ static Coord Floor(const Vec3T& xyz) { return Coord(nanovdb::Floor(xyz[0]), nanovdb::Floor(xyz[1]), nanovdb::Floor(xyz[2])); }
1500 
1501  /// @brief Return a hash key derived from the existing coordinates.
1502  /// @details The hash function is originally taken from the SIGGRAPH paper:
1503  /// "VDB: High-resolution sparse volumes with dynamic topology"
1504  /// and the prime numbers are modified based on the ACM Transactions on Graphics paper:
1505  /// "Real-time 3D reconstruction at scale using voxel hashing" (the second number had a typo!)
1506  template<int Log2N = 3 + 4 + 5>
1507  __hostdev__ uint32_t hash() const { return ((1 << Log2N) - 1) & (mVec[0] * 73856093 ^ mVec[1] * 19349669 ^ mVec[2] * 83492791); }
1508 
1509  /// @brief Return the octant of this Coord
1510  //__hostdev__ size_t octant() const { return (uint32_t(mVec[0])>>31) | ((uint32_t(mVec[1])>>31)<<1) | ((uint32_t(mVec[2])>>31)<<2); }
1511  __hostdev__ uint8_t octant() const { return (uint8_t(bool(mVec[0] & (1u << 31)))) |
1512  (uint8_t(bool(mVec[1] & (1u << 31))) << 1) |
1513  (uint8_t(bool(mVec[2] & (1u << 31))) << 2); }
1514 
1515  /// @brief Return a single precision floating-point vector of this coordinate
1516  __hostdev__ inline Vec3<float> asVec3s() const;
1517 
1518  /// @brief Return a double precision floating-point vector of this coordinate
1519  __hostdev__ inline Vec3<double> asVec3d() const;
1520 
1521  // returns a copy of itself, so it mimics the behaviour of Vec3<T>::round()
1522  __hostdev__ inline Coord round() const { return *this; }
1523 }; // Coord class
1524 
1525 // ----------------------------> Vec3 <--------------------------------------
1526 
1527 /// @brief A simple vector class with three components, similar to openvdb::math::Vec3
1528 template<typename T>
1529 class Vec3
1530 {
1531  T mVec[3];
1532 
1533 public:
1534  static const int SIZE = 3;
1535  static const int size = 3; // in openvdb::math::Tuple
1536  using ValueType = T;
1537  Vec3() = default;
1538  __hostdev__ explicit Vec3(T x)
1539  : mVec{x, x, x}
1540  {
1541  }
1542  __hostdev__ Vec3(T x, T y, T z)
1543  : mVec{x, y, z}
1544  {
1545  }
1546  template<template<class> class Vec3T, class T2>
1547  __hostdev__ Vec3(const Vec3T<T2>& v)
1548  : mVec{T(v[0]), T(v[1]), T(v[2])}
1549  {
1550  static_assert(Vec3T<T2>::size == size, "expected Vec3T::size==3!");
1551  }
1552  template<typename T2>
1553  __hostdev__ explicit Vec3(const Vec3<T2>& v)
1554  : mVec{T(v[0]), T(v[1]), T(v[2])}
1555  {
1556  }
1557  __hostdev__ explicit Vec3(const Coord& ijk)
1558  : mVec{T(ijk[0]), T(ijk[1]), T(ijk[2])}
1559  {
1560  }
1561  __hostdev__ bool operator==(const Vec3& rhs) const { return mVec[0] == rhs[0] && mVec[1] == rhs[1] && mVec[2] == rhs[2]; }
1562  __hostdev__ bool operator!=(const Vec3& rhs) const { return mVec[0] != rhs[0] || mVec[1] != rhs[1] || mVec[2] != rhs[2]; }
1563  template<template<class> class Vec3T, class T2>
1564  __hostdev__ Vec3& operator=(const Vec3T<T2>& rhs)
1565  {
1566  static_assert(Vec3T<T2>::size == size, "expected Vec3T::size==3!");
1567  mVec[0] = rhs[0];
1568  mVec[1] = rhs[1];
1569  mVec[2] = rhs[2];
1570  return *this;
1571  }
1572  __hostdev__ const T& operator[](int i) const { return mVec[i]; }
1573  __hostdev__ T& operator[](int i) { return mVec[i]; }
1574  template<typename Vec3T>
1575  __hostdev__ T dot(const Vec3T& v) const { return mVec[0] * v[0] + mVec[1] * v[1] + mVec[2] * v[2]; }
1576  template<typename Vec3T>
1577  __hostdev__ Vec3 cross(const Vec3T& v) const
1578  {
1579  return Vec3(mVec[1] * v[2] - mVec[2] * v[1],
1580  mVec[2] * v[0] - mVec[0] * v[2],
1581  mVec[0] * v[1] - mVec[1] * v[0]);
1582  }
1584  {
1585  return mVec[0] * mVec[0] + mVec[1] * mVec[1] + mVec[2] * mVec[2]; // 5 flops
1586  }
1587  __hostdev__ T length() const { return Sqrt(this->lengthSqr()); }
1588  __hostdev__ Vec3 operator-() const { return Vec3(-mVec[0], -mVec[1], -mVec[2]); }
1589  __hostdev__ Vec3 operator*(const Vec3& v) const { return Vec3(mVec[0] * v[0], mVec[1] * v[1], mVec[2] * v[2]); }
1590  __hostdev__ Vec3 operator/(const Vec3& v) const { return Vec3(mVec[0] / v[0], mVec[1] / v[1], mVec[2] / v[2]); }
1591  __hostdev__ Vec3 operator+(const Vec3& v) const { return Vec3(mVec[0] + v[0], mVec[1] + v[1], mVec[2] + v[2]); }
1592  __hostdev__ Vec3 operator-(const Vec3& v) const { return Vec3(mVec[0] - v[0], mVec[1] - v[1], mVec[2] - v[2]); }
1593  __hostdev__ Vec3 operator+(const Coord& ijk) const { return Vec3(mVec[0] + ijk[0], mVec[1] + ijk[1], mVec[2] + ijk[2]); }
1594  __hostdev__ Vec3 operator-(const Coord& ijk) const { return Vec3(mVec[0] - ijk[0], mVec[1] - ijk[1], mVec[2] - ijk[2]); }
1595  __hostdev__ Vec3 operator*(const T& s) const { return Vec3(s * mVec[0], s * mVec[1], s * mVec[2]); }
1596  __hostdev__ Vec3 operator/(const T& s) const { return (T(1) / s) * (*this); }
1598  {
1599  mVec[0] += v[0];
1600  mVec[1] += v[1];
1601  mVec[2] += v[2];
1602  return *this;
1603  }
1605  {
1606  mVec[0] += T(ijk[0]);
1607  mVec[1] += T(ijk[1]);
1608  mVec[2] += T(ijk[2]);
1609  return *this;
1610  }
1612  {
1613  mVec[0] -= v[0];
1614  mVec[1] -= v[1];
1615  mVec[2] -= v[2];
1616  return *this;
1617  }
1619  {
1620  mVec[0] -= T(ijk[0]);
1621  mVec[1] -= T(ijk[1]);
1622  mVec[2] -= T(ijk[2]);
1623  return *this;
1624  }
1626  {
1627  mVec[0] *= s;
1628  mVec[1] *= s;
1629  mVec[2] *= s;
1630  return *this;
1631  }
1632  __hostdev__ Vec3& operator/=(const T& s) { return (*this) *= T(1) / s; }
1633  __hostdev__ Vec3& normalize() { return (*this) /= this->length(); }
1634  /// @brief Perform a component-wise minimum with the other Coord.
1636  {
1637  if (other[0] < mVec[0])
1638  mVec[0] = other[0];
1639  if (other[1] < mVec[1])
1640  mVec[1] = other[1];
1641  if (other[2] < mVec[2])
1642  mVec[2] = other[2];
1643  return *this;
1644  }
1645 
1646  /// @brief Perform a component-wise maximum with the other Coord.
1648  {
1649  if (other[0] > mVec[0])
1650  mVec[0] = other[0];
1651  if (other[1] > mVec[1])
1652  mVec[1] = other[1];
1653  if (other[2] > mVec[2])
1654  mVec[2] = other[2];
1655  return *this;
1656  }
1657  /// @brief Return the smallest vector component
1659  {
1660  return mVec[0] < mVec[1] ? (mVec[0] < mVec[2] ? mVec[0] : mVec[2]) : (mVec[1] < mVec[2] ? mVec[1] : mVec[2]);
1661  }
1662  /// @brief Return the largest vector component
1664  {
1665  return mVec[0] > mVec[1] ? (mVec[0] > mVec[2] ? mVec[0] : mVec[2]) : (mVec[1] > mVec[2] ? mVec[1] : mVec[2]);
1666  }
1667  /// @brief Round each component if this Vec<T> up to its integer value
1668  /// @return Return an integer Coord
1669  __hostdev__ Coord floor() const { return Coord(Floor(mVec[0]), Floor(mVec[1]), Floor(mVec[2])); }
1670  /// @brief Round each component if this Vec<T> down to its integer value
1671  /// @return Return an integer Coord
1672  __hostdev__ Coord ceil() const { return Coord(Ceil(mVec[0]), Ceil(mVec[1]), Ceil(mVec[2])); }
1673  /// @brief Round each component if this Vec<T> to its closest integer value
1674  /// @return Return an integer Coord
1676  {
1677  if constexpr(is_same<T, float>::value) {
1678  return Coord(Floor(mVec[0] + 0.5f), Floor(mVec[1] + 0.5f), Floor(mVec[2] + 0.5f));
1679  } else if constexpr(is_same<T, int>::value) {
1680  return Coord(mVec[0], mVec[1], mVec[2]);
1681  } else {
1682  return Coord(Floor(mVec[0] + 0.5), Floor(mVec[1] + 0.5), Floor(mVec[2] + 0.5));
1683  }
1684  }
1685 
1686  /// @brief return a non-const raw constant pointer to array of three vector components
1687  __hostdev__ T* asPointer() { return mVec; }
1688  /// @brief return a const raw constant pointer to array of three vector components
1689  __hostdev__ const T* asPointer() const { return mVec; }
1690 }; // Vec3<T>
1691 
1692 template<typename T1, typename T2>
1693 __hostdev__ inline Vec3<T2> operator*(T1 scalar, const Vec3<T2>& vec)
1694 {
1695  return Vec3<T2>(scalar * vec[0], scalar * vec[1], scalar * vec[2]);
1696 }
1697 template<typename T1, typename T2>
1698 __hostdev__ inline Vec3<T2> operator/(T1 scalar, const Vec3<T2>& vec)
1699 {
1700  return Vec3<T2>(scalar / vec[0], scalar / vec[1], scalar / vec[2]);
1701 }
1702 
1703 //using Vec3R = Vec3<double>;// deprecated
1710 
1711 /// @brief Return a single precision floating-point vector of this coordinate
1713 {
1714  return Vec3f(float(mVec[0]), float(mVec[1]), float(mVec[2]));
1715 }
1716 
1717 /// @brief Return a double precision floating-point vector of this coordinate
1719 {
1720  return Vec3d(double(mVec[0]), double(mVec[1]), double(mVec[2]));
1721 }
1722 
1723 // ----------------------------> Vec4 <--------------------------------------
1724 
1725 /// @brief A simple vector class with four components, similar to openvdb::math::Vec4
1726 template<typename T>
1727 class Vec4
1728 {
1729  T mVec[4];
1730 
1731 public:
1732  static const int SIZE = 4;
1733  static const int size = 4;
1734  using ValueType = T;
1735  Vec4() = default;
1736  __hostdev__ explicit Vec4(T x)
1737  : mVec{x, x, x, x}
1738  {
1739  }
1740  __hostdev__ Vec4(T x, T y, T z, T w)
1741  : mVec{x, y, z, w}
1742  {
1743  }
1744  template<typename T2>
1745  __hostdev__ explicit Vec4(const Vec4<T2>& v)
1746  : mVec{T(v[0]), T(v[1]), T(v[2]), T(v[3])}
1747  {
1748  }
1749  template<template<class> class Vec4T, class T2>
1750  __hostdev__ Vec4(const Vec4T<T2>& v)
1751  : mVec{T(v[0]), T(v[1]), T(v[2]), T(v[3])}
1752  {
1753  static_assert(Vec4T<T2>::size == size, "expected Vec4T::size==4!");
1754  }
1755  __hostdev__ bool operator==(const Vec4& rhs) const { return mVec[0] == rhs[0] && mVec[1] == rhs[1] && mVec[2] == rhs[2] && mVec[3] == rhs[3]; }
1756  __hostdev__ bool operator!=(const Vec4& rhs) const { return mVec[0] != rhs[0] || mVec[1] != rhs[1] || mVec[2] != rhs[2] || mVec[3] != rhs[3]; }
1757  template<template<class> class Vec4T, class T2>
1758  __hostdev__ Vec4& operator=(const Vec4T<T2>& rhs)
1759  {
1760  static_assert(Vec4T<T2>::size == size, "expected Vec4T::size==4!");
1761  mVec[0] = rhs[0];
1762  mVec[1] = rhs[1];
1763  mVec[2] = rhs[2];
1764  mVec[3] = rhs[3];
1765  return *this;
1766  }
1767 
1768  __hostdev__ const T& operator[](int i) const { return mVec[i]; }
1769  __hostdev__ T& operator[](int i) { return mVec[i]; }
1770  template<typename Vec4T>
1771  __hostdev__ T dot(const Vec4T& v) const { return mVec[0] * v[0] + mVec[1] * v[1] + mVec[2] * v[2] + mVec[3] * v[3]; }
1773  {
1774  return mVec[0] * mVec[0] + mVec[1] * mVec[1] + mVec[2] * mVec[2] + mVec[3] * mVec[3]; // 7 flops
1775  }
1776  __hostdev__ T length() const { return Sqrt(this->lengthSqr()); }
1777  __hostdev__ Vec4 operator-() const { return Vec4(-mVec[0], -mVec[1], -mVec[2], -mVec[3]); }
1778  __hostdev__ Vec4 operator*(const Vec4& v) const { return Vec4(mVec[0] * v[0], mVec[1] * v[1], mVec[2] * v[2], mVec[3] * v[3]); }
1779  __hostdev__ Vec4 operator/(const Vec4& v) const { return Vec4(mVec[0] / v[0], mVec[1] / v[1], mVec[2] / v[2], mVec[3] / v[3]); }
1780  __hostdev__ Vec4 operator+(const Vec4& v) const { return Vec4(mVec[0] + v[0], mVec[1] + v[1], mVec[2] + v[2], mVec[3] + v[3]); }
1781  __hostdev__ Vec4 operator-(const Vec4& v) const { return Vec4(mVec[0] - v[0], mVec[1] - v[1], mVec[2] - v[2], mVec[3] - v[3]); }
1782  __hostdev__ Vec4 operator*(const T& s) const { return Vec4(s * mVec[0], s * mVec[1], s * mVec[2], s * mVec[3]); }
1783  __hostdev__ Vec4 operator/(const T& s) const { return (T(1) / s) * (*this); }
1785  {
1786  mVec[0] += v[0];
1787  mVec[1] += v[1];
1788  mVec[2] += v[2];
1789  mVec[3] += v[3];
1790  return *this;
1791  }
1793  {
1794  mVec[0] -= v[0];
1795  mVec[1] -= v[1];
1796  mVec[2] -= v[2];
1797  mVec[3] -= v[3];
1798  return *this;
1799  }
1801  {
1802  mVec[0] *= s;
1803  mVec[1] *= s;
1804  mVec[2] *= s;
1805  mVec[3] *= s;
1806  return *this;
1807  }
1808  __hostdev__ Vec4& operator/=(const T& s) { return (*this) *= T(1) / s; }
1809  __hostdev__ Vec4& normalize() { return (*this) /= this->length(); }
1810  /// @brief Perform a component-wise minimum with the other Coord.
1812  {
1813  if (other[0] < mVec[0])
1814  mVec[0] = other[0];
1815  if (other[1] < mVec[1])
1816  mVec[1] = other[1];
1817  if (other[2] < mVec[2])
1818  mVec[2] = other[2];
1819  if (other[3] < mVec[3])
1820  mVec[3] = other[3];
1821  return *this;
1822  }
1823 
1824  /// @brief Perform a component-wise maximum with the other Coord.
1826  {
1827  if (other[0] > mVec[0])
1828  mVec[0] = other[0];
1829  if (other[1] > mVec[1])
1830  mVec[1] = other[1];
1831  if (other[2] > mVec[2])
1832  mVec[2] = other[2];
1833  if (other[3] > mVec[3])
1834  mVec[3] = other[3];
1835  return *this;
1836  }
1837 }; // Vec4<T>
1838 
1839 template<typename T1, typename T2>
1840 __hostdev__ inline Vec4<T2> operator*(T1 scalar, const Vec4<T2>& vec)
1841 {
1842  return Vec4<T2>(scalar * vec[0], scalar * vec[1], scalar * vec[2], scalar * vec[3]);
1843 }
1844 template<typename T1, typename T2>
1845 __hostdev__ inline Vec4<T2> operator/(T1 scalar, const Vec4<T2>& vec)
1846 {
1847  return Vec4<T2>(scalar / vec[0], scalar / vec[1], scalar / vec[2], scalar / vec[3]);
1848 }
1849 
1854 
1855 
1856 // --------------------------> Rgba8 <------------------------------------
1857 
1858 /// @brief 8-bit red, green, blue, alpha packed into 32 bit unsigned int
1859 class Rgba8
1860 {
1861  union
1862  {
1863  uint8_t c[4]; // 4 integer color channels of red, green, blue and alpha components.
1864  uint32_t packed; // 32 bit packed representation
1865  } mData;
1866 
1867 public:
1868  static const int SIZE = 4;
1869  using ValueType = uint8_t;
1870 
1871  /// @brief Default copy constructor
1872  Rgba8(const Rgba8&) = default;
1873 
1874  /// @brief Default move constructor
1875  Rgba8(Rgba8&&) = default;
1876 
1877  /// @brief Default move assignment operator
1878  /// @return non-const reference to this instance
1879  Rgba8& operator=(Rgba8&&) = default;
1880 
1881  /// @brief Default copy assignment operator
1882  /// @return non-const reference to this instance
1883  Rgba8& operator=(const Rgba8&) = default;
1884 
1885  /// @brief Default ctor initializes all channels to zero
1887  : mData{{0, 0, 0, 0}}
1888  {
1889  static_assert(sizeof(uint32_t) == sizeof(Rgba8), "Unexpected sizeof");
1890  }
1891 
1892  /// @brief integer r,g,b,a ctor where alpha channel defaults to opaque
1893  /// @note all values should be in the range 0u to 255u
1894  __hostdev__ Rgba8(uint8_t r, uint8_t g, uint8_t b, uint8_t a = 255u)
1895  : mData{{r, g, b, a}}
1896  {
1897  }
1898 
1899  /// @brief @brief ctor where all channels are initialized to the same value
1900  /// @note value should be in the range 0u to 255u
1901  explicit __hostdev__ Rgba8(uint8_t v)
1902  : mData{{v, v, v, v}}
1903  {
1904  }
1905 
1906  /// @brief floating-point r,g,b,a ctor where alpha channel defaults to opaque
1907  /// @note all values should be in the range 0.0f to 1.0f
1908  __hostdev__ Rgba8(float r, float g, float b, float a = 1.0f)
1909  : mData{{static_cast<uint8_t>(0.5f + r * 255.0f), // round floats to nearest integers
1910  static_cast<uint8_t>(0.5f + g * 255.0f), // double {{}} is needed due to union
1911  static_cast<uint8_t>(0.5f + b * 255.0f),
1912  static_cast<uint8_t>(0.5f + a * 255.0f)}}
1913  {
1914  }
1915 
1916  /// @brief Vec3f r,g,b ctor (alpha channel it set to 1)
1917  /// @note all values should be in the range 0.0f to 1.0f
1919  : Rgba8(rgb[0], rgb[1], rgb[2])
1920  {
1921  }
1922 
1923  /// @brief Vec4f r,g,b,a ctor
1924  /// @note all values should be in the range 0.0f to 1.0f
1925  __hostdev__ Rgba8(const Vec4f& rgba)
1926  : Rgba8(rgba[0], rgba[1], rgba[2], rgba[3])
1927  {
1928  }
1929 
1930  __hostdev__ bool operator< (const Rgba8& rhs) const { return mData.packed < rhs.mData.packed; }
1931  __hostdev__ bool operator==(const Rgba8& rhs) const { return mData.packed == rhs.mData.packed; }
1932  __hostdev__ float lengthSqr() const
1933  {
1934  return 0.0000153787005f * (float(mData.c[0]) * mData.c[0] +
1935  float(mData.c[1]) * mData.c[1] +
1936  float(mData.c[2]) * mData.c[2]); //1/255^2
1937  }
1938  __hostdev__ float length() const { return sqrtf(this->lengthSqr()); }
1939  /// @brief return n'th color channel as a float in the range 0 to 1
1940  __hostdev__ float asFloat(int n) const { return 0.003921569f*float(mData.c[n]); }// divide by 255
1941  __hostdev__ const uint8_t& operator[](int n) const { return mData.c[n]; }
1942  __hostdev__ uint8_t& operator[](int n) { return mData.c[n]; }
1943  __hostdev__ const uint32_t& packed() const { return mData.packed; }
1944  __hostdev__ uint32_t& packed() { return mData.packed; }
1945  __hostdev__ const uint8_t& r() const { return mData.c[0]; }
1946  __hostdev__ const uint8_t& g() const { return mData.c[1]; }
1947  __hostdev__ const uint8_t& b() const { return mData.c[2]; }
1948  __hostdev__ const uint8_t& a() const { return mData.c[3]; }
1949  __hostdev__ uint8_t& r() { return mData.c[0]; }
1950  __hostdev__ uint8_t& g() { return mData.c[1]; }
1951  __hostdev__ uint8_t& b() { return mData.c[2]; }
1952  __hostdev__ uint8_t& a() { return mData.c[3]; }
1953  __hostdev__ operator Vec3f() const {
1954  return Vec3f(this->asFloat(0), this->asFloat(1), this->asFloat(2));
1955  }
1956  __hostdev__ operator Vec4f() const {
1957  return Vec4f(this->asFloat(0), this->asFloat(1), this->asFloat(2), this->asFloat(3));
1958  }
1959 }; // Rgba8
1960 
1961 using PackedRGBA8 = Rgba8; // for backwards compatibility
1962 
1963 // ----------------------------> TensorTraits <--------------------------------------
1964 
1965 template<typename T, int Rank = (is_specialization<T, Vec3>::value || is_specialization<T, Vec4>::value || is_same<T, Rgba8>::value) ? 1 : 0>
1967 
1968 template<typename T>
1969 struct TensorTraits<T, 0>
1970 {
1971  static const int Rank = 0; // i.e. scalar
1972  static const bool IsScalar = true;
1973  static const bool IsVector = false;
1974  static const int Size = 1;
1975  using ElementType = T;
1976  static T scalar(const T& s) { return s; }
1977 };
1978 
1979 template<typename T>
1980 struct TensorTraits<T, 1>
1981 {
1982  static const int Rank = 1; // i.e. vector
1983  static const bool IsScalar = false;
1984  static const bool IsVector = true;
1985  static const int Size = T::SIZE;
1986  using ElementType = typename T::ValueType;
1987  static ElementType scalar(const T& v) { return v.length(); }
1988 };
1989 
1990 // ----------------------------> FloatTraits <--------------------------------------
1991 
1992 template<typename T, int = sizeof(typename TensorTraits<T>::ElementType)>
1994 {
1995  using FloatType = float;
1996 };
1997 
1998 template<typename T>
1999 struct FloatTraits<T, 8>
2000 {
2001  using FloatType = double;
2002 };
2003 
2004 template<>
2005 struct FloatTraits<bool, 1>
2006 {
2007  using FloatType = bool;
2008 };
2009 
2010 template<>
2011 struct FloatTraits<ValueIndex, 1> // size of empty class in C++ is 1 byte and not 0 byte
2012 {
2013  using FloatType = uint64_t;
2014 };
2015 
2016 template<>
2017 struct FloatTraits<ValueIndexMask, 1> // size of empty class in C++ is 1 byte and not 0 byte
2018 {
2019  using FloatType = uint64_t;
2020 };
2021 
2022 template<>
2023 struct FloatTraits<ValueOnIndex, 1> // size of empty class in C++ is 1 byte and not 0 byte
2024 {
2025  using FloatType = uint64_t;
2026 };
2027 
2028 template<>
2029 struct FloatTraits<ValueOnIndexMask, 1> // size of empty class in C++ is 1 byte and not 0 byte
2030 {
2031  using FloatType = uint64_t;
2032 };
2033 
2034 template<>
2035 struct FloatTraits<ValueMask, 1> // size of empty class in C++ is 1 byte and not 0 byte
2036 {
2037  using FloatType = bool;
2038 };
2039 
2040 template<>
2041 struct FloatTraits<Point, 1> // size of empty class in C++ is 1 byte and not 0 byte
2042 {
2043  using FloatType = double;
2044 };
2045 
2046 // ----------------------------> mapping BuildType -> GridType <--------------------------------------
2047 
2048 /// @brief Maps from a templated build type to a GridType enum
2049 template<typename BuildT>
2051 {
2052  if constexpr(is_same<BuildT, float>::value) { // resolved at compile-time
2053  return GridType::Float;
2054  } else if constexpr(is_same<BuildT, double>::value) {
2055  return GridType::Double;
2056  } else if constexpr(is_same<BuildT, int16_t>::value) {
2057  return GridType::Int16;
2058  } else if constexpr(is_same<BuildT, int32_t>::value) {
2059  return GridType::Int32;
2060  } else if constexpr(is_same<BuildT, int64_t>::value) {
2061  return GridType::Int64;
2062  } else if constexpr(is_same<BuildT, Vec3f>::value) {
2063  return GridType::Vec3f;
2064  } else if constexpr(is_same<BuildT, Vec3d>::value) {
2065  return GridType::Vec3d;
2066  } else if constexpr(is_same<BuildT, uint32_t>::value) {
2067  return GridType::UInt32;
2068  } else if constexpr(is_same<BuildT, ValueMask>::value) {
2069  return GridType::Mask;
2070  } else if constexpr(is_same<BuildT, Half>::value) {
2071  return GridType::Half;
2072  } else if constexpr(is_same<BuildT, ValueIndex>::value) {
2073  return GridType::Index;
2074  } else if constexpr(is_same<BuildT, ValueOnIndex>::value) {
2075  return GridType::OnIndex;
2076  } else if constexpr(is_same<BuildT, ValueIndexMask>::value) {
2077  return GridType::IndexMask;
2078  } else if constexpr(is_same<BuildT, ValueOnIndexMask>::value) {
2079  return GridType::OnIndexMask;
2080  } else if constexpr(is_same<BuildT, bool>::value) {
2081  return GridType::Boolean;
2082  } else if constexpr(is_same<BuildT, Rgba8>::value) {
2083  return GridType::RGBA8;
2084  } else if (is_same<BuildT, Fp4>::value) {
2085  return GridType::Fp4;
2086  } else if constexpr(is_same<BuildT, Fp8>::value) {
2087  return GridType::Fp8;
2088  } else if constexpr(is_same<BuildT, Fp16>::value) {
2089  return GridType::Fp16;
2090  } else if constexpr(is_same<BuildT, FpN>::value) {
2091  return GridType::FpN;
2092  } else if constexpr(is_same<BuildT, Vec4f>::value) {
2093  return GridType::Vec4f;
2094  } else if constexpr(is_same<BuildT, Vec4d>::value) {
2095  return GridType::Vec4d;
2096  } else if (is_same<BuildT, Point>::value) {
2097  return GridType::PointIndex;
2098  } else if constexpr(is_same<BuildT, Vec3u8>::value) {
2099  return GridType::Vec3u8;
2100  } else if constexpr(is_same<BuildT, Vec3u16>::value) {
2101  return GridType::Vec3u16;
2102  }
2103  return GridType::Unknown;
2104 }
2105 
2106 // ----------------------------> mapping BuildType -> GridClass <--------------------------------------
2107 
2108 /// @brief Maps from a templated build type to a GridClass enum
2109 template<typename BuildT>
2111 {
2113  return GridClass::Topology;
2114  } else if (BuildTraits<BuildT>::is_index) {
2115  return GridClass::IndexGrid;
2116  } else if (is_same<BuildT, Rgba8>::value) {
2117  return GridClass::VoxelVolume;
2118  } else if (is_same<BuildT, Point>::value) {
2119  return GridClass::PointIndex;
2120  }
2121  return defaultClass;
2122 }
2123 
2124 // ----------------------------> matMult <--------------------------------------
2125 
2126 /// @brief Multiply a 3x3 matrix and a 3d vector using 32bit floating point arithmetics
2127 /// @note This corresponds to a linear mapping, e.g. scaling, rotation etc.
2128 /// @tparam Vec3T Template type of the input and output 3d vectors
2129 /// @param mat pointer to an array of floats with the 3x3 matrix
2130 /// @param xyz input vector to be multiplied by the matrix
2131 /// @return result of matrix-vector multiplication, i.e. mat x xyz
2132 template<typename Vec3T>
2133 __hostdev__ inline Vec3T matMult(const float* mat, const Vec3T& xyz)
2134 {
2135  return Vec3T(fmaf(static_cast<float>(xyz[0]), mat[0], fmaf(static_cast<float>(xyz[1]), mat[1], static_cast<float>(xyz[2]) * mat[2])),
2136  fmaf(static_cast<float>(xyz[0]), mat[3], fmaf(static_cast<float>(xyz[1]), mat[4], static_cast<float>(xyz[2]) * mat[5])),
2137  fmaf(static_cast<float>(xyz[0]), mat[6], fmaf(static_cast<float>(xyz[1]), mat[7], static_cast<float>(xyz[2]) * mat[8]))); // 6 fmaf + 3 mult = 9 flops
2138 }
2139 
2140 /// @brief Multiply a 3x3 matrix and a 3d vector using 64bit floating point arithmetics
2141 /// @note This corresponds to a linear mapping, e.g. scaling, rotation etc.
2142 /// @tparam Vec3T Template type of the input and output 3d vectors
2143 /// @param mat pointer to an array of floats with the 3x3 matrix
2144 /// @param xyz input vector to be multiplied by the matrix
2145 /// @return result of matrix-vector multiplication, i.e. mat x xyz
2146 template<typename Vec3T>
2147 __hostdev__ inline Vec3T matMult(const double* mat, const Vec3T& xyz)
2148 {
2149  return Vec3T(fma(static_cast<double>(xyz[0]), mat[0], fma(static_cast<double>(xyz[1]), mat[1], static_cast<double>(xyz[2]) * mat[2])),
2150  fma(static_cast<double>(xyz[0]), mat[3], fma(static_cast<double>(xyz[1]), mat[4], static_cast<double>(xyz[2]) * mat[5])),
2151  fma(static_cast<double>(xyz[0]), mat[6], fma(static_cast<double>(xyz[1]), mat[7], static_cast<double>(xyz[2]) * mat[8]))); // 6 fmaf + 3 mult = 9 flops
2152 }
2153 
2154 /// @brief Multiply a 3x3 matrix to a 3d vector and add another 3d vector using 32bit floating point arithmetics
2155 /// @note This corresponds to an affine transformation, i.e a linear mapping followed by a translation. e.g. scale/rotation and translation
2156 /// @tparam Vec3T Template type of the input and output 3d vectors
2157 /// @param mat pointer to an array of floats with the 3x3 matrix
2158 /// @param vec 3d vector to be added AFTER the matrix multiplication
2159 /// @param xyz input vector to be multiplied by the matrix and a translated by @c vec
2160 /// @return result of affine transformation, i.e. (mat x xyz) + vec
2161 template<typename Vec3T>
2162 __hostdev__ inline Vec3T matMult(const float* mat, const float* vec, const Vec3T& xyz)
2163 {
2164  return Vec3T(fmaf(static_cast<float>(xyz[0]), mat[0], fmaf(static_cast<float>(xyz[1]), mat[1], fmaf(static_cast<float>(xyz[2]), mat[2], vec[0]))),
2165  fmaf(static_cast<float>(xyz[0]), mat[3], fmaf(static_cast<float>(xyz[1]), mat[4], fmaf(static_cast<float>(xyz[2]), mat[5], vec[1]))),
2166  fmaf(static_cast<float>(xyz[0]), mat[6], fmaf(static_cast<float>(xyz[1]), mat[7], fmaf(static_cast<float>(xyz[2]), mat[8], vec[2])))); // 9 fmaf = 9 flops
2167 }
2168 
2169 /// @brief Multiply a 3x3 matrix to a 3d vector and add another 3d vector using 64bit floating point arithmetics
2170 /// @note This corresponds to an affine transformation, i.e a linear mapping followed by a translation. e.g. scale/rotation and translation
2171 /// @tparam Vec3T Template type of the input and output 3d vectors
2172 /// @param mat pointer to an array of floats with the 3x3 matrix
2173 /// @param vec 3d vector to be added AFTER the matrix multiplication
2174 /// @param xyz input vector to be multiplied by the matrix and a translated by @c vec
2175 /// @return result of affine transformation, i.e. (mat x xyz) + vec
2176 template<typename Vec3T>
2177 __hostdev__ inline Vec3T matMult(const double* mat, const double* vec, const Vec3T& xyz)
2178 {
2179  return Vec3T(fma(static_cast<double>(xyz[0]), mat[0], fma(static_cast<double>(xyz[1]), mat[1], fma(static_cast<double>(xyz[2]), mat[2], vec[0]))),
2180  fma(static_cast<double>(xyz[0]), mat[3], fma(static_cast<double>(xyz[1]), mat[4], fma(static_cast<double>(xyz[2]), mat[5], vec[1]))),
2181  fma(static_cast<double>(xyz[0]), mat[6], fma(static_cast<double>(xyz[1]), mat[7], fma(static_cast<double>(xyz[2]), mat[8], vec[2])))); // 9 fma = 9 flops
2182 }
2183 
2184 /// @brief Multiply the transposed of a 3x3 matrix and a 3d vector using 32bit floating point arithmetics
2185 /// @note This corresponds to an inverse linear mapping, e.g. inverse scaling, inverse rotation etc.
2186 /// @tparam Vec3T Template type of the input and output 3d vectors
2187 /// @param mat pointer to an array of floats with the 3x3 matrix
2188 /// @param xyz input vector to be multiplied by the transposed matrix
2189 /// @return result of matrix-vector multiplication, i.e. mat^T x xyz
2190 template<typename Vec3T>
2191 __hostdev__ inline Vec3T matMultT(const float* mat, const Vec3T& xyz)
2192 {
2193  return Vec3T(fmaf(static_cast<float>(xyz[0]), mat[0], fmaf(static_cast<float>(xyz[1]), mat[3], static_cast<float>(xyz[2]) * mat[6])),
2194  fmaf(static_cast<float>(xyz[0]), mat[1], fmaf(static_cast<float>(xyz[1]), mat[4], static_cast<float>(xyz[2]) * mat[7])),
2195  fmaf(static_cast<float>(xyz[0]), mat[2], fmaf(static_cast<float>(xyz[1]), mat[5], static_cast<float>(xyz[2]) * mat[8]))); // 6 fmaf + 3 mult = 9 flops
2196 }
2197 
2198 /// @brief Multiply the transposed of a 3x3 matrix and a 3d vector using 64bit floating point arithmetics
2199 /// @note This corresponds to an inverse linear mapping, e.g. inverse scaling, inverse rotation etc.
2200 /// @tparam Vec3T Template type of the input and output 3d vectors
2201 /// @param mat pointer to an array of floats with the 3x3 matrix
2202 /// @param xyz input vector to be multiplied by the transposed matrix
2203 /// @return result of matrix-vector multiplication, i.e. mat^T x xyz
2204 template<typename Vec3T>
2205 __hostdev__ inline Vec3T matMultT(const double* mat, const Vec3T& xyz)
2206 {
2207  return Vec3T(fma(static_cast<double>(xyz[0]), mat[0], fma(static_cast<double>(xyz[1]), mat[3], static_cast<double>(xyz[2]) * mat[6])),
2208  fma(static_cast<double>(xyz[0]), mat[1], fma(static_cast<double>(xyz[1]), mat[4], static_cast<double>(xyz[2]) * mat[7])),
2209  fma(static_cast<double>(xyz[0]), mat[2], fma(static_cast<double>(xyz[1]), mat[5], static_cast<double>(xyz[2]) * mat[8]))); // 6 fmaf + 3 mult = 9 flops
2210 }
2211 
2212 template<typename Vec3T>
2213 __hostdev__ inline Vec3T matMultT(const float* mat, const float* vec, const Vec3T& xyz)
2214 {
2215  return Vec3T(fmaf(static_cast<float>(xyz[0]), mat[0], fmaf(static_cast<float>(xyz[1]), mat[3], fmaf(static_cast<float>(xyz[2]), mat[6], vec[0]))),
2216  fmaf(static_cast<float>(xyz[0]), mat[1], fmaf(static_cast<float>(xyz[1]), mat[4], fmaf(static_cast<float>(xyz[2]), mat[7], vec[1]))),
2217  fmaf(static_cast<float>(xyz[0]), mat[2], fmaf(static_cast<float>(xyz[1]), mat[5], fmaf(static_cast<float>(xyz[2]), mat[8], vec[2])))); // 9 fmaf = 9 flops
2218 }
2219 
2220 template<typename Vec3T>
2221 __hostdev__ inline Vec3T matMultT(const double* mat, const double* vec, const Vec3T& xyz)
2222 {
2223  return Vec3T(fma(static_cast<double>(xyz[0]), mat[0], fma(static_cast<double>(xyz[1]), mat[3], fma(static_cast<double>(xyz[2]), mat[6], vec[0]))),
2224  fma(static_cast<double>(xyz[0]), mat[1], fma(static_cast<double>(xyz[1]), mat[4], fma(static_cast<double>(xyz[2]), mat[7], vec[1]))),
2225  fma(static_cast<double>(xyz[0]), mat[2], fma(static_cast<double>(xyz[1]), mat[5], fma(static_cast<double>(xyz[2]), mat[8], vec[2])))); // 9 fma = 9 flops
2226 }
2227 
2228 // ----------------------------> BBox <-------------------------------------
2229 
2230 // Base-class for static polymorphism (cannot be constructed directly)
2231 template<typename Vec3T>
2232 struct BaseBBox
2233 {
2234  Vec3T mCoord[2];
2235  __hostdev__ bool operator==(const BaseBBox& rhs) const { return mCoord[0] == rhs.mCoord[0] && mCoord[1] == rhs.mCoord[1]; };
2236  __hostdev__ bool operator!=(const BaseBBox& rhs) const { return mCoord[0] != rhs.mCoord[0] || mCoord[1] != rhs.mCoord[1]; };
2237  __hostdev__ const Vec3T& operator[](int i) const { return mCoord[i]; }
2238  __hostdev__ Vec3T& operator[](int i) { return mCoord[i]; }
2239  __hostdev__ Vec3T& min() { return mCoord[0]; }
2240  __hostdev__ Vec3T& max() { return mCoord[1]; }
2241  __hostdev__ const Vec3T& min() const { return mCoord[0]; }
2242  __hostdev__ const Vec3T& max() const { return mCoord[1]; }
2243  __hostdev__ BaseBBox& translate(const Vec3T& xyz)
2244  {
2245  mCoord[0] += xyz;
2246  mCoord[1] += xyz;
2247  return *this;
2248  }
2249  /// @brief Expand this bounding box to enclose point @c xyz.
2250  __hostdev__ BaseBBox& expand(const Vec3T& xyz)
2251  {
2252  mCoord[0].minComponent(xyz);
2253  mCoord[1].maxComponent(xyz);
2254  return *this;
2255  }
2256 
2257  /// @brief Expand this bounding box to enclose the given bounding box.
2259  {
2260  mCoord[0].minComponent(bbox[0]);
2261  mCoord[1].maxComponent(bbox[1]);
2262  return *this;
2263  }
2264 
2265  /// @brief Intersect this bounding box with the given bounding box.
2267  {
2268  mCoord[0].maxComponent(bbox[0]);
2269  mCoord[1].minComponent(bbox[1]);
2270  return *this;
2271  }
2272 
2273  //__hostdev__ BaseBBox expandBy(typename Vec3T::ValueType padding) const
2274  //{
2275  // return BaseBBox(mCoord[0].offsetBy(-padding),mCoord[1].offsetBy(padding));
2276  //}
2277  __hostdev__ bool isInside(const Vec3T& xyz)
2278  {
2279  if (xyz[0] < mCoord[0][0] || xyz[1] < mCoord[0][1] || xyz[2] < mCoord[0][2])
2280  return false;
2281  if (xyz[0] > mCoord[1][0] || xyz[1] > mCoord[1][1] || xyz[2] > mCoord[1][2])
2282  return false;
2283  return true;
2284  }
2285 
2286 protected:
2288  __hostdev__ BaseBBox(const Vec3T& min, const Vec3T& max)
2289  : mCoord{min, max}
2290  {
2291  }
2292 }; // BaseBBox
2293 
2294 template<typename Vec3T, bool = is_floating_point<typename Vec3T::ValueType>::value>
2295 struct BBox;
2296 
2297 /// @brief Partial template specialization for floating point coordinate types.
2298 ///
2299 /// @note Min is inclusive and max is exclusive. If min = max the dimension of
2300 /// the bounding box is zero and therefore it is also empty.
2301 template<typename Vec3T>
2302 struct BBox<Vec3T, true> : public BaseBBox<Vec3T>
2303 {
2304  using Vec3Type = Vec3T;
2305  using ValueType = typename Vec3T::ValueType;
2306  static_assert(is_floating_point<ValueType>::value, "Expected a floating point coordinate type");
2308  using BaseT::mCoord;
2309  /// @brief Default construction sets BBox to an empty bbox
2311  : BaseT(Vec3T( Maximum<typename Vec3T::ValueType>::value()),
2312  Vec3T(-Maximum<typename Vec3T::ValueType>::value()))
2313  {
2314  }
2315  __hostdev__ BBox(const Vec3T& min, const Vec3T& max)
2316  : BaseT(min, max)
2317  {
2318  }
2319  __hostdev__ BBox(const Coord& min, const Coord& max)
2320  : BaseT(Vec3T(ValueType(min[0]), ValueType(min[1]), ValueType(min[2])),
2321  Vec3T(ValueType(max[0] + 1), ValueType(max[1] + 1), ValueType(max[2] + 1)))
2322  {
2323  }
2324  __hostdev__ static BBox createCube(const Coord& min, typename Coord::ValueType dim)
2325  {
2326  return BBox(min, min.offsetBy(dim));
2327  }
2328 
2330  : BBox(bbox[0], bbox[1])
2331  {
2332  }
2333  __hostdev__ bool empty() const { return mCoord[0][0] >= mCoord[1][0] ||
2334  mCoord[0][1] >= mCoord[1][1] ||
2335  mCoord[0][2] >= mCoord[1][2]; }
2336  __hostdev__ operator bool() const { return mCoord[0][0] < mCoord[1][0] &&
2337  mCoord[0][1] < mCoord[1][1] &&
2338  mCoord[0][2] < mCoord[1][2]; }
2339  __hostdev__ Vec3T dim() const { return *this ? this->max() - this->min() : Vec3T(0); }
2340  __hostdev__ bool isInside(const Vec3T& p) const
2341  {
2342  return p[0] > mCoord[0][0] && p[1] > mCoord[0][1] && p[2] > mCoord[0][2] &&
2343  p[0] < mCoord[1][0] && p[1] < mCoord[1][1] && p[2] < mCoord[1][2];
2344  }
2345 
2346 }; // BBox<Vec3T, true>
2347 
2348 /// @brief Partial template specialization for integer coordinate types
2349 ///
2350 /// @note Both min and max are INCLUDED in the bbox so dim = max - min + 1. So,
2351 /// if min = max the bounding box contains exactly one point and dim = 1!
2352 template<typename CoordT>
2353 struct BBox<CoordT, false> : public BaseBBox<CoordT>
2354 {
2355  static_assert(is_same<int, typename CoordT::ValueType>::value, "Expected \"int\" coordinate type");
2357  using BaseT::mCoord;
2358  /// @brief Iterator over the domain covered by a BBox
2359  /// @details z is the fastest-moving coordinate.
2360  class Iterator
2361  {
2362  const BBox& mBBox;
2363  CoordT mPos;
2364 
2365  public:
2367  : mBBox(b)
2368  , mPos(b.min())
2369  {
2370  }
2371  __hostdev__ Iterator(const BBox& b, const Coord& p)
2372  : mBBox(b)
2373  , mPos(p)
2374  {
2375  }
2377  {
2378  if (mPos[2] < mBBox[1][2]) { // this is the most common case
2379  ++mPos[2];// increment z
2380  } else if (mPos[1] < mBBox[1][1]) {
2381  mPos[2] = mBBox[0][2];// reset z
2382  ++mPos[1];// increment y
2383  } else if (mPos[0] <= mBBox[1][0]) {
2384  mPos[2] = mBBox[0][2];// reset z
2385  mPos[1] = mBBox[0][1];// reset y
2386  ++mPos[0];// increment x
2387  }
2388  return *this;
2389  }
2390  __hostdev__ Iterator operator++(int)
2391  {
2392  auto tmp = *this;
2393  ++(*this);
2394  return tmp;
2395  }
2396  __hostdev__ bool operator==(const Iterator& rhs) const
2397  {
2398  NANOVDB_ASSERT(mBBox == rhs.mBBox);
2399  return mPos == rhs.mPos;
2400  }
2401  __hostdev__ bool operator!=(const Iterator& rhs) const
2402  {
2403  NANOVDB_ASSERT(mBBox == rhs.mBBox);
2404  return mPos != rhs.mPos;
2405  }
2406  __hostdev__ bool operator<(const Iterator& rhs) const
2407  {
2408  NANOVDB_ASSERT(mBBox == rhs.mBBox);
2409  return mPos < rhs.mPos;
2410  }
2411  __hostdev__ bool operator<=(const Iterator& rhs) const
2412  {
2413  NANOVDB_ASSERT(mBBox == rhs.mBBox);
2414  return mPos <= rhs.mPos;
2415  }
2416  /// @brief Return @c true if the iterator still points to a valid coordinate.
2417  __hostdev__ operator bool() const { return mPos <= mBBox[1]; }
2418  __hostdev__ const CoordT& operator*() const { return mPos; }
2419  }; // Iterator
2420  __hostdev__ Iterator begin() const { return Iterator{*this}; }
2421  __hostdev__ Iterator end() const { return Iterator{*this, CoordT(mCoord[1][0]+1, mCoord[0][1], mCoord[0][2])}; }
2423  : BaseT(CoordT::max(), CoordT::min())
2424  {
2425  }
2426  __hostdev__ BBox(const CoordT& min, const CoordT& max)
2427  : BaseT(min, max)
2428  {
2429  }
2430 
2431  template<typename SplitT>
2432  __hostdev__ BBox(BBox& other, const SplitT&)
2433  : BaseT(other.mCoord[0], other.mCoord[1])
2434  {
2435  NANOVDB_ASSERT(this->is_divisible());
2436  const int n = MaxIndex(this->dim());
2437  mCoord[1][n] = (mCoord[0][n] + mCoord[1][n]) >> 1;
2438  other.mCoord[0][n] = mCoord[1][n] + 1;
2439  }
2440 
2441  __hostdev__ static BBox createCube(const CoordT& min, typename CoordT::ValueType dim)
2442  {
2443  return BBox(min, min.offsetBy(dim - 1));
2444  }
2445 
2446  __hostdev__ static BBox createCube(typename CoordT::ValueType min, typename CoordT::ValueType max)
2447  {
2448  return BBox(CoordT(min), CoordT(max));
2449  }
2450 
2451  __hostdev__ bool is_divisible() const { return mCoord[0][0] < mCoord[1][0] &&
2452  mCoord[0][1] < mCoord[1][1] &&
2453  mCoord[0][2] < mCoord[1][2]; }
2454  /// @brief Return true if this bounding box is empty, e.g. uninitialized
2455  __hostdev__ bool empty() const { return mCoord[0][0] > mCoord[1][0] ||
2456  mCoord[0][1] > mCoord[1][1] ||
2457  mCoord[0][2] > mCoord[1][2]; }
2458  /// @brief Convert this BBox to boolean true if it is not empty
2459  __hostdev__ operator bool() const { return mCoord[0][0] <= mCoord[1][0] &&
2460  mCoord[0][1] <= mCoord[1][1] &&
2461  mCoord[0][2] <= mCoord[1][2]; }
2462  __hostdev__ CoordT dim() const { return *this ? this->max() - this->min() + Coord(1) : Coord(0); }
2463  __hostdev__ uint64_t volume() const
2464  {
2465  auto d = this->dim();
2466  return uint64_t(d[0]) * uint64_t(d[1]) * uint64_t(d[2]);
2467  }
2468  __hostdev__ bool isInside(const CoordT& p) const { return !(CoordT::lessThan(p, this->min()) || CoordT::lessThan(this->max(), p)); }
2469  /// @brief Return @c true if the given bounding box is inside this bounding box.
2470  __hostdev__ bool isInside(const BBox& b) const
2471  {
2472  return !(CoordT::lessThan(b.min(), this->min()) || CoordT::lessThan(this->max(), b.max()));
2473  }
2474 
2475  /// @brief Return @c true if the given bounding box overlaps with this bounding box.
2476  __hostdev__ bool hasOverlap(const BBox& b) const
2477  {
2478  return !(CoordT::lessThan(this->max(), b.min()) || CoordT::lessThan(b.max(), this->min()));
2479  }
2480 
2481  /// @warning This converts a CoordBBox into a floating-point bounding box which implies that max += 1 !
2482  template<typename RealT = double>
2484  {
2485  static_assert(is_floating_point<RealT>::value, "CoordBBox::asReal: Expected a floating point coordinate");
2486  return BBox<Vec3<RealT>>(Vec3<RealT>(RealT(mCoord[0][0]), RealT(mCoord[0][1]), RealT(mCoord[0][2])),
2487  Vec3<RealT>(RealT(mCoord[1][0] + 1), RealT(mCoord[1][1] + 1), RealT(mCoord[1][2] + 1)));
2488  }
2489  /// @brief Return a new instance that is expanded by the specified padding.
2490  __hostdev__ BBox expandBy(typename CoordT::ValueType padding) const
2491  {
2492  return BBox(mCoord[0].offsetBy(-padding), mCoord[1].offsetBy(padding));
2493  }
2494 
2495  /// @brief @brief transform this coordinate bounding box by the specified map
2496  /// @param map mapping of index to world coordinates
2497  /// @return world bounding box
2498  template<typename Map>
2500  {
2501  const Vec3d tmp = map.applyMap(Vec3d(mCoord[0][0], mCoord[0][1], mCoord[0][2]));
2502  BBox<Vec3d> bbox(tmp, tmp);
2503  bbox.expand(map.applyMap(Vec3d(mCoord[0][0], mCoord[0][1], mCoord[1][2])));
2504  bbox.expand(map.applyMap(Vec3d(mCoord[0][0], mCoord[1][1], mCoord[0][2])));
2505  bbox.expand(map.applyMap(Vec3d(mCoord[1][0], mCoord[0][1], mCoord[0][2])));
2506  bbox.expand(map.applyMap(Vec3d(mCoord[1][0], mCoord[1][1], mCoord[0][2])));
2507  bbox.expand(map.applyMap(Vec3d(mCoord[1][0], mCoord[0][1], mCoord[1][2])));
2508  bbox.expand(map.applyMap(Vec3d(mCoord[0][0], mCoord[1][1], mCoord[1][2])));
2509  bbox.expand(map.applyMap(Vec3d(mCoord[1][0], mCoord[1][1], mCoord[1][2])));
2510  return bbox;
2511  }
2512 
2513 #if defined(__CUDACC__) // the following functions only run on the GPU!
2514  __device__ inline BBox& expandAtomic(const CoordT& ijk)
2515  {
2516  mCoord[0].minComponentAtomic(ijk);
2517  mCoord[1].maxComponentAtomic(ijk);
2518  return *this;
2519  }
2520  __device__ inline BBox& expandAtomic(const BBox& bbox)
2521  {
2522  mCoord[0].minComponentAtomic(bbox[0]);
2523  mCoord[1].maxComponentAtomic(bbox[1]);
2524  return *this;
2525  }
2526  __device__ inline BBox& intersectAtomic(const BBox& bbox)
2527  {
2528  mCoord[0].maxComponentAtomic(bbox[0]);
2529  mCoord[1].minComponentAtomic(bbox[1]);
2530  return *this;
2531  }
2532 #endif
2533 }; // BBox<CoordT, false>
2534 
2537 
2538 // -------------------> Find lowest and highest bit in a word <----------------------------
2539 
2540 /// @brief Returns the index of the lowest, i.e. least significant, on bit in the specified 32 bit word
2541 ///
2542 /// @warning Assumes that at least one bit is set in the word, i.e. @a v != uint32_t(0)!
2544 __hostdev__ static inline uint32_t FindLowestOn(uint32_t v)
2545 {
2546  NANOVDB_ASSERT(v);
2547 #if (defined(__CUDA_ARCH__) || defined(__HIP__)) && defined(NANOVDB_USE_INTRINSICS)
2548  return __ffs(v) - 1; // one based indexing
2549 #elif defined(_MSC_VER) && defined(NANOVDB_USE_INTRINSICS)
2550  unsigned long index;
2551  _BitScanForward(&index, v);
2552  return static_cast<uint32_t>(index);
2553 #elif (defined(__GNUC__) || defined(__clang__)) && defined(NANOVDB_USE_INTRINSICS)
2554  return static_cast<uint32_t>(__builtin_ctzl(v));
2555 #else
2556  //NANO_WARNING("Using software implementation for FindLowestOn(uint32_t v)")
2557  static const unsigned char DeBruijn[32] = {
2558  0, 1, 28, 2, 29, 14, 24, 3, 30, 22, 20, 15, 25, 17, 4, 8, 31, 27, 13, 23, 21, 19, 16, 7, 26, 12, 18, 6, 11, 5, 10, 9};
2559 // disable unary minus on unsigned warning
2560 #if defined(_MSC_VER) && !defined(__NVCC__)
2561 #pragma warning(push)
2562 #pragma warning(disable : 4146)
2563 #endif
2564  return DeBruijn[uint32_t((v & -v) * 0x077CB531U) >> 27];
2565 #if defined(_MSC_VER) && !defined(__NVCC__)
2566 #pragma warning(pop)
2567 #endif
2568 
2569 #endif
2570 }
2571 
2572 /// @brief Returns the index of the highest, i.e. most significant, on bit in the specified 32 bit word
2573 ///
2574 /// @warning Assumes that at least one bit is set in the word, i.e. @a v != uint32_t(0)!
2576 __hostdev__ static inline uint32_t FindHighestOn(uint32_t v)
2577 {
2578  NANOVDB_ASSERT(v);
2579 #if (defined(__CUDA_ARCH__) || defined(__HIP__)) && defined(NANOVDB_USE_INTRINSICS)
2580  return sizeof(uint32_t) * 8 - 1 - __clz(v); // Return the number of consecutive high-order zero bits in a 32-bit integer.
2581 #elif defined(_MSC_VER) && defined(NANOVDB_USE_INTRINSICS)
2582  unsigned long index;
2583  _BitScanReverse(&index, v);
2584  return static_cast<uint32_t>(index);
2585 #elif (defined(__GNUC__) || defined(__clang__)) && defined(NANOVDB_USE_INTRINSICS)
2586  return sizeof(unsigned long) * 8 - 1 - __builtin_clzl(v);
2587 #else
2588  //NANO_WARNING("Using software implementation for FindHighestOn(uint32_t)")
2589  static const unsigned char DeBruijn[32] = {
2590  0, 9, 1, 10, 13, 21, 2, 29, 11, 14, 16, 18, 22, 25, 3, 30,
2591  8, 12, 20, 28, 15, 17, 24, 7, 19, 27, 23, 6, 26, 5, 4, 31};
2592  v |= v >> 1; // first round down to one less than a power of 2
2593  v |= v >> 2;
2594  v |= v >> 4;
2595  v |= v >> 8;
2596  v |= v >> 16;
2597  return DeBruijn[uint32_t(v * 0x07C4ACDDU) >> 27];
2598 #endif
2599 }
2600 
2601 /// @brief Returns the index of the lowest, i.e. least significant, on bit in the specified 64 bit word
2602 ///
2603 /// @warning Assumes that at least one bit is set in the word, i.e. @a v != uint32_t(0)!
2605 __hostdev__ static inline uint32_t FindLowestOn(uint64_t v)
2606 {
2607  NANOVDB_ASSERT(v);
2608 #if (defined(__CUDA_ARCH__) || defined(__HIP__)) && defined(NANOVDB_USE_INTRINSICS)
2609  return __ffsll(static_cast<unsigned long long int>(v)) - 1; // one based indexing
2610 #elif defined(_MSC_VER) && defined(NANOVDB_USE_INTRINSICS)
2611  unsigned long index;
2612  _BitScanForward64(&index, v);
2613  return static_cast<uint32_t>(index);
2614 #elif (defined(__GNUC__) || defined(__clang__)) && defined(NANOVDB_USE_INTRINSICS)
2615  return static_cast<uint32_t>(__builtin_ctzll(v));
2616 #else
2617  //NANO_WARNING("Using software implementation for FindLowestOn(uint64_t)")
2618  static const unsigned char DeBruijn[64] = {
2619  0, 1, 2, 53, 3, 7, 54, 27, 4, 38, 41, 8, 34, 55, 48, 28,
2620  62, 5, 39, 46, 44, 42, 22, 9, 24, 35, 59, 56, 49, 18, 29, 11,
2621  63, 52, 6, 26, 37, 40, 33, 47, 61, 45, 43, 21, 23, 58, 17, 10,
2622  51, 25, 36, 32, 60, 20, 57, 16, 50, 31, 19, 15, 30, 14, 13, 12,
2623  };
2624 // disable unary minus on unsigned warning
2625 #if defined(_MSC_VER) && !defined(__NVCC__)
2626 #pragma warning(push)
2627 #pragma warning(disable : 4146)
2628 #endif
2629  return DeBruijn[uint64_t((v & -v) * UINT64_C(0x022FDD63CC95386D)) >> 58];
2630 #if defined(_MSC_VER) && !defined(__NVCC__)
2631 #pragma warning(pop)
2632 #endif
2633 
2634 #endif
2635 }
2636 
2637 /// @brief Returns the index of the highest, i.e. most significant, on bit in the specified 64 bit word
2638 ///
2639 /// @warning Assumes that at least one bit is set in the word, i.e. @a v != uint32_t(0)!
2641 __hostdev__ static inline uint32_t FindHighestOn(uint64_t v)
2642 {
2643  NANOVDB_ASSERT(v);
2644 #if (defined(__CUDA_ARCH__) || defined(__HIP__)) && defined(NANOVDB_USE_INTRINSICS)
2645  return sizeof(unsigned long) * 8 - 1 - __clzll(static_cast<unsigned long long int>(v));
2646 #elif defined(_MSC_VER) && defined(NANOVDB_USE_INTRINSICS)
2647  unsigned long index;
2648  _BitScanReverse64(&index, v);
2649  return static_cast<uint32_t>(index);
2650 #elif (defined(__GNUC__) || defined(__clang__)) && defined(NANOVDB_USE_INTRINSICS)
2651  return sizeof(unsigned long) * 8 - 1 - __builtin_clzll(v);
2652 #else
2653  const uint32_t* p = reinterpret_cast<const uint32_t*>(&v);
2654  return p[1] ? 32u + FindHighestOn(p[1]) : FindHighestOn(p[0]);
2655 #endif
2656 }
2657 
2658 // ----------------------------> CountOn <--------------------------------------
2659 
2660 /// @return Number of bits that are on in the specified 64-bit word
2662 __hostdev__ inline uint32_t CountOn(uint64_t v)
2663 {
2664 #if (defined(__CUDA_ARCH__) || defined(__HIP__)) && defined(NANOVDB_USE_INTRINSICS)
2665  //#warning Using popcll for CountOn
2666  return __popcll(v);
2667 // __popcnt64 intrinsic support was added in VS 2019 16.8
2668 #elif defined(_MSC_VER) && defined(_M_X64) && (_MSC_VER >= 1928) && defined(NANOVDB_USE_INTRINSICS)
2669  //#warning Using popcnt64 for CountOn
2670  return uint32_t(__popcnt64(v));
2671 #elif (defined(__GNUC__) || defined(__clang__)) && defined(NANOVDB_USE_INTRINSICS)
2672  //#warning Using builtin_popcountll for CountOn
2673  return __builtin_popcountll(v);
2674 #else // use software implementation
2675  //NANO_WARNING("Using software implementation for CountOn")
2676  v = v - ((v >> 1) & uint64_t(0x5555555555555555));
2677  v = (v & uint64_t(0x3333333333333333)) + ((v >> 2) & uint64_t(0x3333333333333333));
2678  return (((v + (v >> 4)) & uint64_t(0xF0F0F0F0F0F0F0F)) * uint64_t(0x101010101010101)) >> 56;
2679 #endif
2680 }
2681 
2682 // ----------------------------> BitFlags <--------------------------------------
2683 
2684 template<int N>
2685 struct BitArray;
2686 template<>
2687 struct BitArray<8>
2688 {
2689  uint8_t mFlags{0};
2690 };
2691 template<>
2692 struct BitArray<16>
2693 {
2694  uint16_t mFlags{0};
2695 };
2696 template<>
2697 struct BitArray<32>
2698 {
2699  uint32_t mFlags{0};
2700 };
2701 template<>
2702 struct BitArray<64>
2703 {
2704  uint64_t mFlags{0};
2705 };
2706 
2707 template<int N>
2708 class BitFlags : public BitArray<N>
2709 {
2710 protected:
2711  using BitArray<N>::mFlags;
2712 
2713 public:
2714  using Type = decltype(mFlags);
2716  BitFlags(std::initializer_list<uint8_t> list)
2717  {
2718  for (auto bit : list)
2719  mFlags |= static_cast<Type>(1 << bit);
2720  }
2721  template<typename MaskT>
2722  BitFlags(std::initializer_list<MaskT> list)
2723  {
2724  for (auto mask : list)
2725  mFlags |= static_cast<Type>(mask);
2726  }
2727  __hostdev__ Type data() const { return mFlags; }
2728  __hostdev__ Type& data() { return mFlags; }
2729  __hostdev__ void initBit(std::initializer_list<uint8_t> list)
2730  {
2731  mFlags = 0u;
2732  for (auto bit : list)
2733  mFlags |= static_cast<Type>(1 << bit);
2734  }
2735  template<typename MaskT>
2736  __hostdev__ void initMask(std::initializer_list<MaskT> list)
2737  {
2738  mFlags = 0u;
2739  for (auto mask : list)
2740  mFlags |= static_cast<Type>(mask);
2741  }
2742  //__hostdev__ Type& data() { return mFlags; }
2743  //__hostdev__ Type data() const { return mFlags; }
2744  __hostdev__ Type getFlags() const { return mFlags & (static_cast<Type>(GridFlags::End) - 1u); } // mask out everything except relevant bits
2745 
2746  __hostdev__ void setOn() { mFlags = ~Type(0u); }
2747  __hostdev__ void setOff() { mFlags = Type(0u); }
2748 
2749  __hostdev__ void setBitOn(uint8_t bit) { mFlags |= static_cast<Type>(1 << bit); }
2750  __hostdev__ void setBitOff(uint8_t bit) { mFlags &= ~static_cast<Type>(1 << bit); }
2751 
2752  __hostdev__ void setBitOn(std::initializer_list<uint8_t> list)
2753  {
2754  for (auto bit : list)
2755  mFlags |= static_cast<Type>(1 << bit);
2756  }
2757  __hostdev__ void setBitOff(std::initializer_list<uint8_t> list)
2758  {
2759  for (auto bit : list)
2760  mFlags &= ~static_cast<Type>(1 << bit);
2761  }
2762 
2763  template<typename MaskT>
2764  __hostdev__ void setMaskOn(MaskT mask) { mFlags |= static_cast<Type>(mask); }
2765  template<typename MaskT>
2766  __hostdev__ void setMaskOff(MaskT mask) { mFlags &= ~static_cast<Type>(mask); }
2767 
2768  template<typename MaskT>
2769  __hostdev__ void setMaskOn(std::initializer_list<MaskT> list)
2770  {
2771  for (auto mask : list)
2772  mFlags |= static_cast<Type>(mask);
2773  }
2774  template<typename MaskT>
2775  __hostdev__ void setMaskOff(std::initializer_list<MaskT> list)
2776  {
2777  for (auto mask : list)
2778  mFlags &= ~static_cast<Type>(mask);
2779  }
2780 
2781  __hostdev__ void setBit(uint8_t bit, bool on) { on ? this->setBitOn(bit) : this->setBitOff(bit); }
2782  template<typename MaskT>
2783  __hostdev__ void setMask(MaskT mask, bool on) { on ? this->setMaskOn(mask) : this->setMaskOff(mask); }
2784 
2785  __hostdev__ bool isOn() const { return mFlags == ~Type(0u); }
2786  __hostdev__ bool isOff() const { return mFlags == Type(0u); }
2787  __hostdev__ bool isBitOn(uint8_t bit) const { return 0 != (mFlags & static_cast<Type>(1 << bit)); }
2788  __hostdev__ bool isBitOff(uint8_t bit) const { return 0 == (mFlags & static_cast<Type>(1 << bit)); }
2789  template<typename MaskT>
2790  __hostdev__ bool isMaskOn(MaskT mask) const { return 0 != (mFlags & static_cast<Type>(mask)); }
2791  template<typename MaskT>
2792  __hostdev__ bool isMaskOff(MaskT mask) const { return 0 == (mFlags & static_cast<Type>(mask)); }
2793  /// @brief return true if any of the masks in the list are on
2794  template<typename MaskT>
2795  __hostdev__ bool isMaskOn(std::initializer_list<MaskT> list) const
2796  {
2797  for (auto mask : list)
2798  if (0 != (mFlags & static_cast<Type>(mask)))
2799  return true;
2800  return false;
2801  }
2802  /// @brief return true if any of the masks in the list are off
2803  template<typename MaskT>
2804  __hostdev__ bool isMaskOff(std::initializer_list<MaskT> list) const
2805  {
2806  for (auto mask : list)
2807  if (0 == (mFlags & static_cast<Type>(mask)))
2808  return true;
2809  return false;
2810  }
2811  /// @brief required for backwards compatibility
2813  {
2814  mFlags = n;
2815  return *this;
2816  }
2817 }; // BitFlags<N>
2818 
2819 // ----------------------------> Mask <--------------------------------------
2820 
2821 /// @brief Bit-mask to encode active states and facilitate sequential iterators
2822 /// and a fast codec for I/O compression.
2823 template<uint32_t LOG2DIM>
2824 class Mask
2825 {
2826 public:
2827  static constexpr uint32_t SIZE = 1U << (3 * LOG2DIM); // Number of bits in mask
2828  static constexpr uint32_t WORD_COUNT = SIZE >> 6; // Number of 64 bit words
2829 
2830  /// @brief Return the memory footprint in bytes of this Mask
2831  __hostdev__ static size_t memUsage() { return sizeof(Mask); }
2832 
2833  /// @brief Return the number of bits available in this Mask
2834  __hostdev__ static uint32_t bitCount() { return SIZE; }
2835 
2836  /// @brief Return the number of machine words used by this Mask
2837  __hostdev__ static uint32_t wordCount() { return WORD_COUNT; }
2838 
2839  /// @brief Return the total number of set bits in this Mask
2840  __hostdev__ uint32_t countOn() const
2841  {
2842  uint32_t sum = 0;
2843  for (const uint64_t *w = mWords, *q = w + WORD_COUNT; w != q; ++w)
2844  sum += CountOn(*w);
2845  return sum;
2846  }
2847 
2848  /// @brief Return the number of lower set bits in mask up to but excluding the i'th bit
2849  inline __hostdev__ uint32_t countOn(uint32_t i) const
2850  {
2851  uint32_t n = i >> 6, sum = CountOn(mWords[n] & ((uint64_t(1) << (i & 63u)) - 1u));
2852  for (const uint64_t* w = mWords; n--; ++w)
2853  sum += CountOn(*w);
2854  return sum;
2855  }
2856 
2857  template<bool On>
2858  class Iterator
2859  {
2860  public:
2862  : mPos(Mask::SIZE)
2863  , mParent(nullptr)
2864  {
2865  }
2866  __hostdev__ Iterator(uint32_t pos, const Mask* parent)
2867  : mPos(pos)
2868  , mParent(parent)
2869  {
2870  }
2871  Iterator& operator=(const Iterator&) = default;
2872  __hostdev__ uint32_t operator*() const { return mPos; }
2873  __hostdev__ uint32_t pos() const { return mPos; }
2874  __hostdev__ operator bool() const { return mPos != Mask::SIZE; }
2876  {
2877  mPos = mParent->findNext<On>(mPos + 1);
2878  return *this;
2879  }
2881  {
2882  auto tmp = *this;
2883  ++(*this);
2884  return tmp;
2885  }
2886 
2887  private:
2888  uint32_t mPos;
2889  const Mask* mParent;
2890  }; // Member class Iterator
2891 
2893  {
2894  public:
2896  : mPos(pos)
2897  {
2898  }
2899  DenseIterator& operator=(const DenseIterator&) = default;
2900  __hostdev__ uint32_t operator*() const { return mPos; }
2901  __hostdev__ uint32_t pos() const { return mPos; }
2902  __hostdev__ operator bool() const { return mPos != Mask::SIZE; }
2904  {
2905  ++mPos;
2906  return *this;
2907  }
2909  {
2910  auto tmp = *this;
2911  ++mPos;
2912  return tmp;
2913  }
2914 
2915  private:
2916  uint32_t mPos;
2917  }; // Member class DenseIterator
2918 
2921 
2922  __hostdev__ OnIterator beginOn() const { return OnIterator(this->findFirst<true>(), this); }
2923 
2924  __hostdev__ OffIterator beginOff() const { return OffIterator(this->findFirst<false>(), this); }
2925 
2927 
2928  /// @brief Initialize all bits to zero.
2930  {
2931  for (uint32_t i = 0; i < WORD_COUNT; ++i)
2932  mWords[i] = 0;
2933  }
2935  {
2936  const uint64_t v = on ? ~uint64_t(0) : uint64_t(0);
2937  for (uint32_t i = 0; i < WORD_COUNT; ++i)
2938  mWords[i] = v;
2939  }
2940 
2941  /// @brief Copy constructor
2942  __hostdev__ Mask(const Mask& other)
2943  {
2944  for (uint32_t i = 0; i < WORD_COUNT; ++i)
2945  mWords[i] = other.mWords[i];
2946  }
2947 
2948  /// @brief Return a pointer to the list of words of the bit mask
2949  __hostdev__ uint64_t* words() { return mWords; }
2950  __hostdev__ const uint64_t* words() const { return mWords; }
2951 
2952  /// @brief Assignment operator that works with openvdb::util::NodeMask
2953  template<typename MaskT = Mask>
2955  {
2956  static_assert(sizeof(Mask) == sizeof(MaskT), "Mismatching sizeof");
2957  static_assert(WORD_COUNT == MaskT::WORD_COUNT, "Mismatching word count");
2958  static_assert(LOG2DIM == MaskT::LOG2DIM, "Mismatching LOG2DIM");
2959  auto* src = reinterpret_cast<const uint64_t*>(&other);
2960  for (uint64_t *dst = mWords, *end = dst + WORD_COUNT; dst != end; ++dst)
2961  *dst = *src++;
2962  return *this;
2963  }
2964 
2966  {
2967  memcpy64(mWords, other.mWords, WORD_COUNT);
2968  return *this;
2969  }
2970 
2971  __hostdev__ bool operator==(const Mask& other) const
2972  {
2973  for (uint32_t i = 0; i < WORD_COUNT; ++i) {
2974  if (mWords[i] != other.mWords[i])
2975  return false;
2976  }
2977  return true;
2978  }
2979 
2980  __hostdev__ bool operator!=(const Mask& other) const { return !((*this) == other); }
2981 
2982  /// @brief Return true if the given bit is set.
2983  __hostdev__ bool isOn(uint32_t n) const { return 0 != (mWords[n >> 6] & (uint64_t(1) << (n & 63))); }
2984 
2985  /// @brief Return true if the given bit is NOT set.
2986  __hostdev__ bool isOff(uint32_t n) const { return 0 == (mWords[n >> 6] & (uint64_t(1) << (n & 63))); }
2987 
2988  /// @brief Return true if all the bits are set in this Mask.
2989  __hostdev__ bool isOn() const
2990  {
2991  for (uint32_t i = 0; i < WORD_COUNT; ++i)
2992  if (mWords[i] != ~uint64_t(0))
2993  return false;
2994  return true;
2995  }
2996 
2997  /// @brief Return true if none of the bits are set in this Mask.
2998  __hostdev__ bool isOff() const
2999  {
3000  for (uint32_t i = 0; i < WORD_COUNT; ++i)
3001  if (mWords[i] != uint64_t(0))
3002  return false;
3003  return true;
3004  }
3005 
3006  /// @brief Set the specified bit on.
3007  __hostdev__ void setOn(uint32_t n) { mWords[n >> 6] |= uint64_t(1) << (n & 63); }
3008  /// @brief Set the specified bit off.
3009  __hostdev__ void setOff(uint32_t n) { mWords[n >> 6] &= ~(uint64_t(1) << (n & 63)); }
3010 
3011 #if defined(__CUDACC__) // the following functions only run on the GPU!
3012  __device__ inline void setOnAtomic(uint32_t n)
3013  {
3014  atomicOr(reinterpret_cast<unsigned long long int*>(this) + (n >> 6), 1ull << (n & 63));
3015  }
3016  __device__ inline void setOffAtomic(uint32_t n)
3017  {
3018  atomicAnd(reinterpret_cast<unsigned long long int*>(this) + (n >> 6), ~(1ull << (n & 63)));
3019  }
3020  __device__ inline void setAtomic(uint32_t n, bool on)
3021  {
3022  on ? this->setOnAtomic(n) : this->setOffAtomic(n);
3023  }
3024 #endif
3025  /// @brief Set the specified bit on or off.
3026  __hostdev__ void set(uint32_t n, bool on)
3027  {
3028 #if 1 // switch between branchless
3029  auto& word = mWords[n >> 6];
3030  n &= 63;
3031  word &= ~(uint64_t(1) << n);
3032  word |= uint64_t(on) << n;
3033 #else
3034  on ? this->setOn(n) : this->setOff(n);
3035 #endif
3036  }
3037 
3038  /// @brief Set all bits on
3040  {
3041  for (uint32_t i = 0; i < WORD_COUNT; ++i)
3042  mWords[i] = ~uint64_t(0);
3043  }
3044 
3045  /// @brief Set all bits off
3047  {
3048  for (uint32_t i = 0; i < WORD_COUNT; ++i)
3049  mWords[i] = uint64_t(0);
3050  }
3051 
3052  /// @brief Set all bits off
3053  __hostdev__ void set(bool on)
3054  {
3055  const uint64_t v = on ? ~uint64_t(0) : uint64_t(0);
3056  for (uint32_t i = 0; i < WORD_COUNT; ++i)
3057  mWords[i] = v;
3058  }
3059  /// brief Toggle the state of all bits in the mask
3061  {
3062  uint32_t n = WORD_COUNT;
3063  for (auto* w = mWords; n--; ++w)
3064  *w = ~*w;
3065  }
3066  __hostdev__ void toggle(uint32_t n) { mWords[n >> 6] ^= uint64_t(1) << (n & 63); }
3067 
3068  /// @brief Bitwise intersection
3070  {
3071  uint64_t* w1 = mWords;
3072  const uint64_t* w2 = other.mWords;
3073  for (uint32_t n = WORD_COUNT; n--; ++w1, ++w2)
3074  *w1 &= *w2;
3075  return *this;
3076  }
3077  /// @brief Bitwise union
3079  {
3080  uint64_t* w1 = mWords;
3081  const uint64_t* w2 = other.mWords;
3082  for (uint32_t n = WORD_COUNT; n--; ++w1, ++w2)
3083  *w1 |= *w2;
3084  return *this;
3085  }
3086  /// @brief Bitwise difference
3088  {
3089  uint64_t* w1 = mWords;
3090  const uint64_t* w2 = other.mWords;
3091  for (uint32_t n = WORD_COUNT; n--; ++w1, ++w2)
3092  *w1 &= ~*w2;
3093  return *this;
3094  }
3095  /// @brief Bitwise XOR
3097  {
3098  uint64_t* w1 = mWords;
3099  const uint64_t* w2 = other.mWords;
3100  for (uint32_t n = WORD_COUNT; n--; ++w1, ++w2)
3101  *w1 ^= *w2;
3102  return *this;
3103  }
3104 
3106  template<bool ON>
3107  __hostdev__ uint32_t findFirst() const
3108  {
3109  uint32_t n = 0u;
3110  const uint64_t* w = mWords;
3111  for (; n < WORD_COUNT && !(ON ? *w : ~*w); ++w, ++n)
3112  ;
3113  return n < WORD_COUNT ? (n << 6) + FindLowestOn(ON ? *w : ~*w) : SIZE;
3114  }
3115 
3117  template<bool ON>
3118  __hostdev__ uint32_t findNext(uint32_t start) const
3119  {
3120  uint32_t n = start >> 6; // initiate
3121  if (n >= WORD_COUNT)
3122  return SIZE; // check for out of bounds
3123  uint32_t m = start & 63u;
3124  uint64_t b = ON ? mWords[n] : ~mWords[n];
3125  if (b & (uint64_t(1u) << m))
3126  return start; // simple case: start is on/off
3127  b &= ~uint64_t(0u) << m; // mask out lower bits
3128  while (!b && ++n < WORD_COUNT)
3129  b = ON ? mWords[n] : ~mWords[n]; // find next non-zero word
3130  return b ? (n << 6) + FindLowestOn(b) : SIZE; // catch last word=0
3131  }
3132 
3134  template<bool ON>
3135  __hostdev__ uint32_t findPrev(uint32_t start) const
3136  {
3137  uint32_t n = start >> 6; // initiate
3138  if (n >= WORD_COUNT)
3139  return SIZE; // check for out of bounds
3140  uint32_t m = start & 63u;
3141  uint64_t b = ON ? mWords[n] : ~mWords[n];
3142  if (b & (uint64_t(1u) << m))
3143  return start; // simple case: start is on/off
3144  b &= (uint64_t(1u) << m) - 1u; // mask out higher bits
3145  while (!b && n)
3146  b = ON ? mWords[--n] : ~mWords[--n]; // find previous non-zero word
3147  return b ? (n << 6) + FindHighestOn(b) : SIZE; // catch first word=0
3148  }
3149 
3150 private:
3151  uint64_t mWords[WORD_COUNT];
3152 }; // Mask class
3153 
3154 // ----------------------------> Map <--------------------------------------
3155 
3156 /// @brief Defines an affine transform and its inverse represented as a 3x3 matrix and a vec3 translation
3157 struct Map
3158 { // 264B (not 32B aligned!)
3159  float mMatF[9]; // 9*4B <- 3x3 matrix
3160  float mInvMatF[9]; // 9*4B <- 3x3 matrix
3161  float mVecF[3]; // 3*4B <- translation
3162  float mTaperF; // 4B, placeholder for taper value
3163  double mMatD[9]; // 9*8B <- 3x3 matrix
3164  double mInvMatD[9]; // 9*8B <- 3x3 matrix
3165  double mVecD[3]; // 3*8B <- translation
3166  double mTaperD; // 8B, placeholder for taper value
3167 
3168  /// @brief Default constructor for the identity map
3170  : mMatF{1.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 1.0f}
3171  , mInvMatF{1.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 1.0f}
3172  , mVecF{0.0f, 0.0f, 0.0f}
3173  , mTaperF{1.0f}
3174  , mMatD{1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0}
3175  , mInvMatD{1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0}
3176  , mVecD{0.0, 0.0, 0.0}
3177  , mTaperD{1.0}
3178  {
3179  }
3180  __hostdev__ Map(double s, const Vec3d& t = Vec3d(0.0, 0.0, 0.0))
3181  : mMatF{float(s), 0.0f, 0.0f, 0.0f, float(s), 0.0f, 0.0f, 0.0f, float(s)}
3182  , mInvMatF{1.0f / float(s), 0.0f, 0.0f, 0.0f, 1.0f / float(s), 0.0f, 0.0f, 0.0f, 1.0f / float(s)}
3183  , mVecF{float(t[0]), float(t[1]), float(t[2])}
3184  , mTaperF{1.0f}
3185  , mMatD{s, 0.0, 0.0, 0.0, s, 0.0, 0.0, 0.0, s}
3186  , mInvMatD{1.0 / s, 0.0, 0.0, 0.0, 1.0 / s, 0.0, 0.0, 0.0, 1.0 / s}
3187  , mVecD{t[0], t[1], t[2]}
3188  , mTaperD{1.0}
3189  {
3190  }
3191 
3192  /// @brief Initialize the member data from 3x3 or 4x4 matrices
3193  /// @note This is not _hostdev__ since then MatT=openvdb::Mat4d will produce warnings
3194  template<typename MatT, typename Vec3T>
3195  void set(const MatT& mat, const MatT& invMat, const Vec3T& translate, double taper = 1.0);
3196 
3197  /// @brief Initialize the member data from 4x4 matrices
3198  /// @note The last (4th) row of invMat is actually ignored.
3199  /// This is not _hostdev__ since then Mat4T=openvdb::Mat4d will produce warnings
3200  template<typename Mat4T>
3201  void set(const Mat4T& mat, const Mat4T& invMat, double taper = 1.0) { this->set(mat, invMat, mat[3], taper); }
3202 
3203  template<typename Vec3T>
3204  void set(double scale, const Vec3T& translation, double taper = 1.0);
3205 
3206  /// @brief Apply the forward affine transformation to a vector using 64bit floating point arithmetics.
3207  /// @note Typically this operation is used for the scale, rotation and translation of index -> world mapping
3208  /// @tparam Vec3T Template type of the 3D vector to be mapped
3209  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3210  /// @return Forward mapping for affine transformation, i.e. (mat x ijk) + translation
3211  template<typename Vec3T>
3212  __hostdev__ Vec3T applyMap(const Vec3T& ijk) const { return matMult(mMatD, mVecD, ijk); }
3213 
3214  /// @brief Apply the forward affine transformation to a vector using 32bit floating point arithmetics.
3215  /// @note Typically this operation is used for the scale, rotation and translation of index -> world mapping
3216  /// @tparam Vec3T Template type of the 3D vector to be mapped
3217  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3218  /// @return Forward mapping for affine transformation, i.e. (mat x ijk) + translation
3219  template<typename Vec3T>
3220  __hostdev__ Vec3T applyMapF(const Vec3T& ijk) const { return matMult(mMatF, mVecF, ijk); }
3221 
3222  /// @brief Apply the linear forward 3x3 transformation to an input 3d vector using 64bit floating point arithmetics,
3223  /// e.g. scale and rotation WITHOUT translation.
3224  /// @note Typically this operation is used for scale and rotation from index -> world mapping
3225  /// @tparam Vec3T Template type of the 3D vector to be mapped
3226  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3227  /// @return linear forward 3x3 mapping of the input vector
3228  template<typename Vec3T>
3229  __hostdev__ Vec3T applyJacobian(const Vec3T& ijk) const { return matMult(mMatD, ijk); }
3230 
3231  /// @brief Apply the linear forward 3x3 transformation to an input 3d vector using 32bit floating point arithmetics,
3232  /// e.g. scale and rotation WITHOUT translation.
3233  /// @note Typically this operation is used for scale and rotation from index -> world mapping
3234  /// @tparam Vec3T Template type of the 3D vector to be mapped
3235  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3236  /// @return linear forward 3x3 mapping of the input vector
3237  template<typename Vec3T>
3238  __hostdev__ Vec3T applyJacobianF(const Vec3T& ijk) const { return matMult(mMatF, ijk); }
3239 
3240  /// @brief Apply the inverse affine mapping to a vector using 64bit floating point arithmetics.
3241  /// @note Typically this operation is used for the world -> index mapping
3242  /// @tparam Vec3T Template type of the 3D vector to be mapped
3243  /// @param xyz 3D vector to be mapped - typically floating point world coordinates
3244  /// @return Inverse affine mapping of the input @c xyz i.e. (xyz - translation) x mat^-1
3245  template<typename Vec3T>
3246  __hostdev__ Vec3T applyInverseMap(const Vec3T& xyz) const
3247  {
3248  return matMult(mInvMatD, Vec3T(xyz[0] - mVecD[0], xyz[1] - mVecD[1], xyz[2] - mVecD[2]));
3249  }
3250 
3251  /// @brief Apply the inverse affine mapping to a vector using 32bit floating point arithmetics.
3252  /// @note Typically this operation is used for the world -> index mapping
3253  /// @tparam Vec3T Template type of the 3D vector to be mapped
3254  /// @param xyz 3D vector to be mapped - typically floating point world coordinates
3255  /// @return Inverse affine mapping of the input @c xyz i.e. (xyz - translation) x mat^-1
3256  template<typename Vec3T>
3257  __hostdev__ Vec3T applyInverseMapF(const Vec3T& xyz) const
3258  {
3259  return matMult(mInvMatF, Vec3T(xyz[0] - mVecF[0], xyz[1] - mVecF[1], xyz[2] - mVecF[2]));
3260  }
3261 
3262  /// @brief Apply the linear inverse 3x3 transformation to an input 3d vector using 64bit floating point arithmetics,
3263  /// e.g. inverse scale and inverse rotation WITHOUT translation.
3264  /// @note Typically this operation is used for scale and rotation from world -> index mapping
3265  /// @tparam Vec3T Template type of the 3D vector to be mapped
3266  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3267  /// @return linear inverse 3x3 mapping of the input vector i.e. xyz x mat^-1
3268  template<typename Vec3T>
3269  __hostdev__ Vec3T applyInverseJacobian(const Vec3T& xyz) const { return matMult(mInvMatD, xyz); }
3270 
3271  /// @brief Apply the linear inverse 3x3 transformation to an input 3d vector using 32bit floating point arithmetics,
3272  /// e.g. inverse scale and inverse rotation WITHOUT translation.
3273  /// @note Typically this operation is used for scale and rotation from world -> index mapping
3274  /// @tparam Vec3T Template type of the 3D vector to be mapped
3275  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3276  /// @return linear inverse 3x3 mapping of the input vector i.e. xyz x mat^-1
3277  template<typename Vec3T>
3278  __hostdev__ Vec3T applyInverseJacobianF(const Vec3T& xyz) const { return matMult(mInvMatF, xyz); }
3279 
3280  /// @brief Apply the transposed inverse 3x3 transformation to an input 3d vector using 64bit floating point arithmetics,
3281  /// e.g. inverse scale and inverse rotation WITHOUT translation.
3282  /// @note Typically this operation is used for scale and rotation from world -> index mapping
3283  /// @tparam Vec3T Template type of the 3D vector to be mapped
3284  /// @param ijk 3D vector to be mapped - typically floating point index coordinates
3285  /// @return linear inverse 3x3 mapping of the input vector i.e. xyz x mat^-1
3286  template<typename Vec3T>
3287  __hostdev__ Vec3T applyIJT(const Vec3T& xyz) const { return matMultT(mInvMatD, xyz); }
3288  template<typename Vec3T>
3289  __hostdev__ Vec3T applyIJTF(const Vec3T& xyz) const { return matMultT(mInvMatF, xyz); }
3290 
3291  /// @brief Return a voxels size in each coordinate direction, measured at the origin
3292  __hostdev__ Vec3d getVoxelSize() const { return this->applyMap(Vec3d(1)) - this->applyMap(Vec3d(0)); }
3293 }; // Map
3294 
3295 template<typename MatT, typename Vec3T>
3296 inline void Map::set(const MatT& mat, const MatT& invMat, const Vec3T& translate, double taper)
3297 {
3298  float * mf = mMatF, *vf = mVecF, *mif = mInvMatF;
3299  double *md = mMatD, *vd = mVecD, *mid = mInvMatD;
3300  mTaperF = static_cast<float>(taper);
3301  mTaperD = taper;
3302  for (int i = 0; i < 3; ++i) {
3303  *vd++ = translate[i]; //translation
3304  *vf++ = static_cast<float>(translate[i]); //translation
3305  for (int j = 0; j < 3; ++j) {
3306  *md++ = mat[j][i]; //transposed
3307  *mid++ = invMat[j][i];
3308  *mf++ = static_cast<float>(mat[j][i]); //transposed
3309  *mif++ = static_cast<float>(invMat[j][i]);
3310  }
3311  }
3312 }
3313 
3314 template<typename Vec3T>
3315 inline void Map::set(double dx, const Vec3T& trans, double taper)
3316 {
3317  NANOVDB_ASSERT(dx > 0.0);
3318  const double mat[3][3] = { {dx, 0.0, 0.0}, // row 0
3319  {0.0, dx, 0.0}, // row 1
3320  {0.0, 0.0, dx} }; // row 2
3321  const double idx = 1.0 / dx;
3322  const double invMat[3][3] = { {idx, 0.0, 0.0}, // row 0
3323  {0.0, idx, 0.0}, // row 1
3324  {0.0, 0.0, idx} }; // row 2
3325  this->set(mat, invMat, trans, taper);
3326 }
3327 
3328 // ----------------------------> GridBlindMetaData <--------------------------------------
3329 
3330 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) GridBlindMetaData
3331 { // 288 bytes
3332  static const int MaxNameSize = 256; // due to NULL termination the maximum length is one less!
3333  int64_t mDataOffset; // byte offset to the blind data, relative to this GridBlindMetaData.
3334  uint64_t mValueCount; // number of blind values, e.g. point count
3335  uint32_t mValueSize;// byte size of each value, e.g. 4 if mDataType=Float and 1 if mDataType=Unknown since that amounts to char
3336  GridBlindDataSemantic mSemantic; // semantic meaning of the data.
3338  GridType mDataType; // 4 bytes
3339  char mName[MaxNameSize]; // note this includes the NULL termination
3340  // no padding required for 32 byte alignment
3341 
3342  // disallow copy-construction since methods like blindData and getBlindData uses the this pointer!
3343  GridBlindMetaData(const GridBlindMetaData&) = delete;
3344 
3345  // disallow copy-assignment since methods like blindData and getBlindData uses the this pointer!
3346  const GridBlindMetaData& operator=(const GridBlindMetaData&) = delete;
3347 
3348  __hostdev__ void setBlindData(void* blindData) { mDataOffset = PtrDiff(blindData, this); }
3349 
3350  // unsafe
3351  __hostdev__ const void* blindData() const {return PtrAdd<void>(this, mDataOffset);}
3352 
3353  /// @brief Get a const pointer to the blind data represented by this meta data
3354  /// @tparam BlindDataT Expected value type of the blind data.
3355  /// @return Returns NULL if mGridType!=mapToGridType<BlindDataT>(), else a const point of type BlindDataT.
3356  /// @note Use mDataType=Unknown if BlindDataT is a custom data type unknown to NanoVDB.
3357  template<typename BlindDataT>
3358  __hostdev__ const BlindDataT* getBlindData() const
3359  {
3360  //if (mDataType != mapToGridType<BlindDataT>()) printf("getBlindData mismatch\n");
3361  return mDataType == mapToGridType<BlindDataT>() ? PtrAdd<BlindDataT>(this, mDataOffset) : nullptr;
3362  }
3363 
3364  /// @brief return true if this meta data has a valid combination of semantic, class and value tags
3365  __hostdev__ bool isValid() const
3366  {
3367  auto check = [&]()->bool{
3368  switch (mDataType){
3369  case GridType::Unknown: return mValueSize==1u;// i.e. we encode data as mValueCount chars
3370  case GridType::Float: return mValueSize==4u;
3371  case GridType::Double: return mValueSize==8u;
3372  case GridType::Int16: return mValueSize==2u;
3373  case GridType::Int32: return mValueSize==4u;
3374  case GridType::Int64: return mValueSize==8u;
3375  case GridType::Vec3f: return mValueSize==12u;
3376  case GridType::Vec3d: return mValueSize==24u;
3377  case GridType::Half: return mValueSize==2u;
3378  case GridType::RGBA8: return mValueSize==4u;
3379  case GridType::Fp8: return mValueSize==1u;
3380  case GridType::Fp16: return mValueSize==2u;
3381  case GridType::Vec4f: return mValueSize==16u;
3382  case GridType::Vec4d: return mValueSize==32u;
3383  case GridType::Vec3u8: return mValueSize==3u;
3384  case GridType::Vec3u16: return mValueSize==6u;
3385  default: return true;}// all other combinations are valid
3386  };
3387  return nanovdb::isValid(mDataClass, mSemantic, mDataType) && check();
3388  }
3389 
3390  /// @brief return size in bytes of the blind data represented by this blind meta data
3391  /// @note This size includes possible padding for 32 byte alignment. The actual amount
3392  /// of bind data is mValueCount * mValueSize
3393  __hostdev__ uint64_t blindDataSize() const
3394  {
3395  return AlignUp<NANOVDB_DATA_ALIGNMENT>(mValueCount * mValueSize);
3396  }
3397 }; // GridBlindMetaData
3398 
3399 // ----------------------------> NodeTrait <--------------------------------------
3400 
3401 /// @brief Struct to derive node type from its level in a given
3402 /// grid, tree or root while preserving constness
3403 template<typename GridOrTreeOrRootT, int LEVEL>
3404 struct NodeTrait;
3405 
3406 // Partial template specialization of above Node struct
3407 template<typename GridOrTreeOrRootT>
3408 struct NodeTrait<GridOrTreeOrRootT, 0>
3409 {
3410  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3411  using Type = typename GridOrTreeOrRootT::LeafNodeType;
3412  using type = typename GridOrTreeOrRootT::LeafNodeType;
3413 };
3414 template<typename GridOrTreeOrRootT>
3415 struct NodeTrait<const GridOrTreeOrRootT, 0>
3416 {
3417  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3418  using Type = const typename GridOrTreeOrRootT::LeafNodeType;
3419  using type = const typename GridOrTreeOrRootT::LeafNodeType;
3420 };
3421 
3422 template<typename GridOrTreeOrRootT>
3423 struct NodeTrait<GridOrTreeOrRootT, 1>
3424 {
3425  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3426  using Type = typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType;
3427  using type = typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType;
3428 };
3429 template<typename GridOrTreeOrRootT>
3430 struct NodeTrait<const GridOrTreeOrRootT, 1>
3431 {
3432  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3433  using Type = const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType;
3434  using type = const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType;
3435 };
3436 template<typename GridOrTreeOrRootT>
3437 struct NodeTrait<GridOrTreeOrRootT, 2>
3438 {
3439  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3440  using Type = typename GridOrTreeOrRootT::RootNodeType::ChildNodeType;
3441  using type = typename GridOrTreeOrRootT::RootNodeType::ChildNodeType;
3442 };
3443 template<typename GridOrTreeOrRootT>
3444 struct NodeTrait<const GridOrTreeOrRootT, 2>
3445 {
3446  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3447  using Type = const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType;
3448  using type = const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType;
3449 };
3450 template<typename GridOrTreeOrRootT>
3451 struct NodeTrait<GridOrTreeOrRootT, 3>
3452 {
3453  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3454  using Type = typename GridOrTreeOrRootT::RootNodeType;
3455  using type = typename GridOrTreeOrRootT::RootNodeType;
3456 };
3457 
3458 template<typename GridOrTreeOrRootT>
3459 struct NodeTrait<const GridOrTreeOrRootT, 3>
3460 {
3461  static_assert(GridOrTreeOrRootT::RootNodeType::LEVEL == 3, "Tree depth is not supported");
3462  using Type = const typename GridOrTreeOrRootT::RootNodeType;
3463  using type = const typename GridOrTreeOrRootT::RootNodeType;
3464 };
3465 
3466 // ----------------------------> Froward decelerations of random access methods <--------------------------------------
3467 
3468 template<typename BuildT>
3469 struct GetValue;
3470 template<typename BuildT>
3471 struct SetValue;
3472 template<typename BuildT>
3473 struct SetVoxel;
3474 template<typename BuildT>
3475 struct GetState;
3476 template<typename BuildT>
3477 struct GetDim;
3478 template<typename BuildT>
3479 struct GetLeaf;
3480 template<typename BuildT>
3481 struct ProbeValue;
3482 template<typename BuildT>
3484 
3485 // ----------------------------> Grid <--------------------------------------
3486 
3487 /*
3488  The following class and comment is for internal use only
3489 
3490  Memory layout:
3491 
3492  Grid -> 39 x double (world bbox and affine transformation)
3493  Tree -> Root 3 x ValueType + int32_t + N x Tiles (background,min,max,tileCount + tileCount x Tiles)
3494 
3495  N2 upper InternalNodes each with 2 bit masks, N2 tiles, and min/max values
3496 
3497  N1 lower InternalNodes each with 2 bit masks, N1 tiles, and min/max values
3498 
3499  N0 LeafNodes each with a bit mask, N0 ValueTypes and min/max
3500 
3501  Example layout: ("---" implies it has a custom offset, "..." implies zero or more)
3502  [GridData][TreeData]---[RootData][ROOT TILES...]---[InternalData<5>]---[InternalData<4>]---[LeafData<3>]---[BLINDMETA...]---[BLIND0]---[BLIND1]---etc.
3503 */
3504 
3505 /// @brief Struct with all the member data of the Grid (useful during serialization of an openvdb grid)
3506 ///
3507 /// @note The transform is assumed to be affine (so linear) and have uniform scale! So frustum transforms
3508 /// and non-uniform scaling are not supported (primarily because they complicate ray-tracing in index space)
3509 ///
3510 /// @note No client code should (or can) interface with this struct so it can safely be ignored!
3511 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) GridData
3512 { // sizeof(GridData) = 672B
3513  static const int MaxNameSize = 256; // due to NULL termination the maximum length is one less
3514  uint64_t mMagic; // 8B (0) magic to validate it is valid grid data.
3515  uint64_t mChecksum; // 8B (8). Checksum of grid buffer.
3516  Version mVersion; // 4B (16) major, minor, and patch version numbers
3517  BitFlags<32> mFlags; // 4B (20). flags for grid.
3518  uint32_t mGridIndex; // 4B (24). Index of this grid in the buffer
3519  uint32_t mGridCount; // 4B (28). Total number of grids in the buffer
3520  uint64_t mGridSize; // 8B (32). byte count of this entire grid occupied in the buffer.
3521  char mGridName[MaxNameSize]; // 256B (40)
3522  Map mMap; // 264B (296). affine transformation between index and world space in both single and double precision
3523  BBox<Vec3d> mWorldBBox; // 48B (560). floating-point AABB of active values in WORLD SPACE (2 x 3 doubles)
3524  Vec3d mVoxelSize; // 24B (608). size of a voxel in world units
3525  GridClass mGridClass; // 4B (632).
3526  GridType mGridType; // 4B (636).
3527  int64_t mBlindMetadataOffset; // 8B (640). offset to beginning of GridBlindMetaData structures that follow this grid.
3528  uint32_t mBlindMetadataCount; // 4B (648). count of GridBlindMetaData structures that follow this grid.
3529  uint32_t mData0; // 4B (652)
3530  uint64_t mData1, mData2; // 2x8B (656) padding to 32 B alignment. mData1 is use for the total number of values indexed by an IndexGrid
3531  /// @brief Use this method to initiate most member dat
3533  {
3534  static_assert(8 * 84 == sizeof(GridData), "GridData has unexpected size");
3535  memcpy64(this, &other, 84);
3536  return *this;
3537  }
3538  __hostdev__ void init(std::initializer_list<GridFlags> list = {GridFlags::IsBreadthFirst},
3539  uint64_t gridSize = 0u,
3540  const Map& map = Map(),
3541  GridType gridType = GridType::Unknown,
3542  GridClass gridClass = GridClass::Unknown)
3543  {
3544 #ifdef NANOVDB_USE_NEW_MAGIC_NUMBERS
3545  mMagic = NANOVDB_MAGIC_GRID;
3546 #else
3547  mMagic = NANOVDB_MAGIC_NUMBER;
3548 #endif
3549  mChecksum = ~uint64_t(0);// all 64 bits ON means checksum is disabled
3550  mVersion = Version();
3551  mFlags.initMask(list);
3552  mGridIndex = 0u;
3553  mGridCount = 1u;
3554  mGridSize = gridSize;
3555  mGridName[0] = '\0';
3556  mMap = map;
3557  mWorldBBox = BBox<Vec3d>();// invalid bbox
3558  mVoxelSize = map.getVoxelSize();
3559  mGridClass = gridClass;
3560  mGridType = gridType;
3561  mBlindMetadataOffset = mGridSize; // i.e. no blind data
3562  mBlindMetadataCount = 0u; // i.e. no blind data
3563  mData0 = 0u;
3564  mData1 = 0u; // only used for index and point grids
3565  mData2 = 0u;
3566  }
3567  /// @brief return true if the magic number and the version are both valid
3568  __hostdev__ bool isValid() const {
3569  return mMagic == NANOVDB_MAGIC_GRID || (mMagic == NANOVDB_MAGIC_NUMBER && mVersion.isCompatible());
3570  }
3571  // Set and unset various bit flags
3572  __hostdev__ void setMinMaxOn(bool on = true) { mFlags.setMask(GridFlags::HasMinMax, on); }
3573  __hostdev__ void setBBoxOn(bool on = true) { mFlags.setMask(GridFlags::HasBBox, on); }
3574  __hostdev__ void setLongGridNameOn(bool on = true) { mFlags.setMask(GridFlags::HasLongGridName, on); }
3575  __hostdev__ void setAverageOn(bool on = true) { mFlags.setMask(GridFlags::HasAverage, on); }
3576  __hostdev__ void setStdDeviationOn(bool on = true) { mFlags.setMask(GridFlags::HasStdDeviation, on); }
3577  __hostdev__ bool setGridName(const char* src)
3578  {
3579  char *dst = mGridName, *end = dst + MaxNameSize;
3580  while (*src != '\0' && dst < end - 1)
3581  *dst++ = *src++;
3582  while (dst < end)
3583  *dst++ = '\0';
3584  return *src == '\0'; // returns true if input grid name is NOT longer than MaxNameSize characters
3585  }
3586  // Affine transformations based on double precision
3587  template<typename Vec3T>
3588  __hostdev__ Vec3T applyMap(const Vec3T& xyz) const { return mMap.applyMap(xyz); } // Pos: index -> world
3589  template<typename Vec3T>
3590  __hostdev__ Vec3T applyInverseMap(const Vec3T& xyz) const { return mMap.applyInverseMap(xyz); } // Pos: world -> index
3591  template<typename Vec3T>
3592  __hostdev__ Vec3T applyJacobian(const Vec3T& xyz) const { return mMap.applyJacobian(xyz); } // Dir: index -> world
3593  template<typename Vec3T>
3594  __hostdev__ Vec3T applyInverseJacobian(const Vec3T& xyz) const { return mMap.applyInverseJacobian(xyz); } // Dir: world -> index
3595  template<typename Vec3T>
3596  __hostdev__ Vec3T applyIJT(const Vec3T& xyz) const { return mMap.applyIJT(xyz); }
3597  // Affine transformations based on single precision
3598  template<typename Vec3T>
3599  __hostdev__ Vec3T applyMapF(const Vec3T& xyz) const { return mMap.applyMapF(xyz); } // Pos: index -> world
3600  template<typename Vec3T>
3601  __hostdev__ Vec3T applyInverseMapF(const Vec3T& xyz) const { return mMap.applyInverseMapF(xyz); } // Pos: world -> index
3602  template<typename Vec3T>
3603  __hostdev__ Vec3T applyJacobianF(const Vec3T& xyz) const { return mMap.applyJacobianF(xyz); } // Dir: index -> world
3604  template<typename Vec3T>
3605  __hostdev__ Vec3T applyInverseJacobianF(const Vec3T& xyz) const { return mMap.applyInverseJacobianF(xyz); } // Dir: world -> index
3606  template<typename Vec3T>
3607  __hostdev__ Vec3T applyIJTF(const Vec3T& xyz) const { return mMap.applyIJTF(xyz); }
3608 
3609  // @brief Return a non-const uint8_t pointer to the tree
3610  __hostdev__ uint8_t* treePtr() { return reinterpret_cast<uint8_t*>(this + 1); }// TreeData is always right after GridData
3611  //__hostdev__ TreeData* treePtr() { return reinterpret_cast<TreeData*>(this + 1); }// TreeData is always right after GridData
3612 
3613  // @brief Return a const uint8_t pointer to the tree
3614  __hostdev__ const uint8_t* treePtr() const { return reinterpret_cast<const uint8_t*>(this + 1); }// TreeData is always right after GridData
3615  //__hostdev__ const TreeData* treePtr() const { return reinterpret_cast<const TreeData*>(this + 1); }// TreeData is always right after GridData
3616 
3617  /// @brief Return a non-const uint8_t pointer to the first node at @c LEVEL
3618  /// @tparam LEVEL of the node. LEVEL 0 means leaf node and LEVEL 3 means root node
3619  /// @warning If not nodes exist at @c LEVEL NULL is returned
3620  template <uint32_t LEVEL>
3621  __hostdev__ const uint8_t* nodePtr() const
3622  {
3623  static_assert(LEVEL >= 0 && LEVEL <= 3, "invalid LEVEL template parameter");
3624  auto *treeData = this->treePtr();
3625  auto nodeOffset = *reinterpret_cast<const uint64_t*>(treeData + 8*LEVEL);// skip LEVEL uint64_t
3626  return nodeOffset ? PtrAdd<uint8_t>(treeData, nodeOffset) : nullptr;
3627  }
3628 
3629  /// @brief Return a non-const uint8_t pointer to the first node at @c LEVEL
3630  /// @tparam LEVEL of the node. LEVEL 0 means leaf node and LEVEL 3 means root node
3631  /// @warning If not nodes exist at @c LEVEL NULL is returned
3632  template <uint32_t LEVEL>
3633  __hostdev__ uint8_t* nodePtr(){return const_cast<uint8_t*>(const_cast<const GridData*>(this)->template nodePtr<LEVEL>());}
3634 
3635  /// @brief Returns a const reference to the blindMetaData at the specified linear offset.
3636  ///
3637  /// @warning The linear offset is assumed to be in the valid range
3639  {
3640  NANOVDB_ASSERT(n < mBlindMetadataCount);
3641  return PtrAdd<GridBlindMetaData>(this, mBlindMetadataOffset) + n;
3642  }
3643 
3644  __hostdev__ const char* gridName() const
3645  {
3646  if (mFlags.isMaskOn(GridFlags::HasLongGridName)) {// search for first blind meta data that contains a name
3647  NANOVDB_ASSERT(mBlindMetadataCount > 0);
3648  for (uint32_t i = 0; i < mBlindMetadataCount; ++i) {
3649  const auto* metaData = this->blindMetaData(i);// EXTREMELY important to be a pointer
3650  if (metaData->mDataClass == GridBlindDataClass::GridName) {
3651  NANOVDB_ASSERT(metaData->mDataType == GridType::Unknown);
3652  return metaData->template getBlindData<const char>();
3653  }
3654  }
3655  NANOVDB_ASSERT(false); // should never hit this!
3656  }
3657  return mGridName;
3658  }
3659 
3660  /// @brief Return memory usage in bytes for this class only.
3661  __hostdev__ static uint64_t memUsage() { return sizeof(GridData); }
3662 
3663  /// @brief return AABB of active values in world space
3664  __hostdev__ const BBox<Vec3d>& worldBBox() const { return mWorldBBox; }
3665 
3666  /// @brief return AABB of active values in index space
3667  __hostdev__ const CoordBBox& indexBBox() const {return *(const CoordBBox*)(this->nodePtr<3>());}
3668 
3669  /// @brief return the root table has size
3670  __hostdev__ uint32_t rootTableSize() const {
3671  if (const uint8_t *root = this->nodePtr<3>()) {
3672  return *(const uint32_t*)(root + sizeof(CoordBBox));
3673  }
3674  return 0u;
3675  }
3676 
3677  /// @brief test if the grid is empty, e.i the root table has size 0
3678  /// @return true if this grid contains not data whatsoever
3679  __hostdev__ bool isEmpty() const {return this->rootTableSize() == 0u;}
3680 
3681  /// @brief return true if RootData follows TreeData in memory without any extra padding
3682  /// @details TreeData is always following right after GridData, but the same might not be true for RootData
3683  __hostdev__ bool isRootConnected() const { return *(const uint64_t*)((const char*)(this + 1) + 24) == 64u;}
3684 }; // GridData
3685 
3686 // Forward declaration of accelerated random access class
3687 template<typename BuildT, int LEVEL0 = -1, int LEVEL1 = -1, int LEVEL2 = -1>
3689 
3690 template<typename BuildT>
3692 
3693 /// @brief Highest level of the data structure. Contains a tree and a world->index
3694 /// transform (that currently only supports uniform scaling and translation).
3695 ///
3696 /// @note This the API of this class to interface with client code
3697 template<typename TreeT>
3698 class Grid : public GridData
3699 {
3700 public:
3701  using TreeType = TreeT;
3702  using RootType = typename TreeT::RootType;
3704  using UpperNodeType = typename RootNodeType::ChildNodeType;
3705  using LowerNodeType = typename UpperNodeType::ChildNodeType;
3706  using LeafNodeType = typename RootType::LeafNodeType;
3708  using ValueType = typename TreeT::ValueType;
3709  using BuildType = typename TreeT::BuildType; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
3710  using CoordType = typename TreeT::CoordType;
3712 
3713  /// @brief Disallow constructions, copy and assignment
3714  ///
3715  /// @note Only a Serializer, defined elsewhere, can instantiate this class
3716  Grid(const Grid&) = delete;
3717  Grid& operator=(const Grid&) = delete;
3718  ~Grid() = delete;
3719 
3720  __hostdev__ Version version() const { return DataType::mVersion; }
3721 
3722  __hostdev__ DataType* data() { return reinterpret_cast<DataType*>(this); }
3723 
3724  __hostdev__ const DataType* data() const { return reinterpret_cast<const DataType*>(this); }
3725 
3726  /// @brief Return memory usage in bytes for this class only.
3727  //__hostdev__ static uint64_t memUsage() { return sizeof(GridData); }
3728 
3729  /// @brief Return the memory footprint of the entire grid, i.e. including all nodes and blind data
3730  __hostdev__ uint64_t gridSize() const { return DataType::mGridSize; }
3731 
3732  /// @brief Return index of this grid in the buffer
3733  __hostdev__ uint32_t gridIndex() const { return DataType::mGridIndex; }
3734 
3735  /// @brief Return total number of grids in the buffer
3736  __hostdev__ uint32_t gridCount() const { return DataType::mGridCount; }
3737 
3738  /// @brief @brief Return the total number of values indexed by this IndexGrid
3739  ///
3740  /// @note This method is only defined for IndexGrid = NanoGrid<ValueIndex || ValueOnIndex || ValueIndexMask || ValueOnIndexMask>
3741  template<typename T = BuildType>
3742  __hostdev__ typename enable_if<BuildTraits<T>::is_index, const uint64_t&>::type
3743  valueCount() const { return DataType::mData1; }
3744 
3745  /// @brief @brief Return the total number of points indexed by this PointGrid
3746  ///
3747  /// @note This method is only defined for PointGrid = NanoGrid<Point>
3748  template<typename T = BuildType>
3749  __hostdev__ typename enable_if<is_same<T, Point>::value, const uint64_t&>::type
3750  pointCount() const { return DataType::mData1; }
3751 
3752  /// @brief Return a const reference to the tree
3753  __hostdev__ const TreeT& tree() const { return *reinterpret_cast<const TreeT*>(this->treePtr()); }
3754 
3755  /// @brief Return a non-const reference to the tree
3756  __hostdev__ TreeT& tree() { return *reinterpret_cast<TreeT*>(this->treePtr()); }
3757 
3758  /// @brief Return a new instance of a ReadAccessor used to access values in this grid
3759  __hostdev__ AccessorType getAccessor() const { return AccessorType(this->tree().root()); }
3760 
3761  /// @brief Return a const reference to the size of a voxel in world units
3762  __hostdev__ const Vec3d& voxelSize() const { return DataType::mVoxelSize; }
3763 
3764  /// @brief Return a const reference to the Map for this grid
3765  __hostdev__ const Map& map() const { return DataType::mMap; }
3766 
3767  /// @brief world to index space transformation
3768  template<typename Vec3T>
3769  __hostdev__ Vec3T worldToIndex(const Vec3T& xyz) const { return this->applyInverseMap(xyz); }
3770 
3771  /// @brief index to world space transformation
3772  template<typename Vec3T>
3773  __hostdev__ Vec3T indexToWorld(const Vec3T& xyz) const { return this->applyMap(xyz); }
3774 
3775  /// @brief transformation from index space direction to world space direction
3776  /// @warning assumes dir to be normalized
3777  template<typename Vec3T>
3778  __hostdev__ Vec3T indexToWorldDir(const Vec3T& dir) const { return this->applyJacobian(dir); }
3779 
3780  /// @brief transformation from world space direction to index space direction
3781  /// @warning assumes dir to be normalized
3782  template<typename Vec3T>
3783  __hostdev__ Vec3T worldToIndexDir(const Vec3T& dir) const { return this->applyInverseJacobian(dir); }
3784 
3785  /// @brief transform the gradient from index space to world space.
3786  /// @details Applies the inverse jacobian transform map.
3787  template<typename Vec3T>
3788  __hostdev__ Vec3T indexToWorldGrad(const Vec3T& grad) const { return this->applyIJT(grad); }
3789 
3790  /// @brief world to index space transformation
3791  template<typename Vec3T>
3792  __hostdev__ Vec3T worldToIndexF(const Vec3T& xyz) const { return this->applyInverseMapF(xyz); }
3793 
3794  /// @brief index to world space transformation
3795  template<typename Vec3T>
3796  __hostdev__ Vec3T indexToWorldF(const Vec3T& xyz) const { return this->applyMapF(xyz); }
3797 
3798  /// @brief transformation from index space direction to world space direction
3799  /// @warning assumes dir to be normalized
3800  template<typename Vec3T>
3801  __hostdev__ Vec3T indexToWorldDirF(const Vec3T& dir) const { return this->applyJacobianF(dir); }
3802 
3803  /// @brief transformation from world space direction to index space direction
3804  /// @warning assumes dir to be normalized
3805  template<typename Vec3T>
3806  __hostdev__ Vec3T worldToIndexDirF(const Vec3T& dir) const { return this->applyInverseJacobianF(dir); }
3807 
3808  /// @brief Transforms the gradient from index space to world space.
3809  /// @details Applies the inverse jacobian transform map.
3810  template<typename Vec3T>
3811  __hostdev__ Vec3T indexToWorldGradF(const Vec3T& grad) const { return DataType::applyIJTF(grad); }
3812 
3813  /// @brief Computes a AABB of active values in world space
3814  //__hostdev__ const BBox<Vec3d>& worldBBox() const { return DataType::mWorldBBox; }
3815 
3816  /// @brief Computes a AABB of active values in index space
3817  ///
3818  /// @note This method is returning a floating point bounding box and not a CoordBBox. This makes
3819  /// it more useful for clipping rays.
3820  //__hostdev__ const BBox<CoordType>& indexBBox() const { return this->tree().bbox(); }
3821 
3822  /// @brief Return the total number of active voxels in this tree.
3823  __hostdev__ uint64_t activeVoxelCount() const { return this->tree().activeVoxelCount(); }
3824 
3825  /// @brief Methods related to the classification of this grid
3826  __hostdev__ bool isValid() const { return DataType::isValid(); }
3827  __hostdev__ const GridType& gridType() const { return DataType::mGridType; }
3828  __hostdev__ const GridClass& gridClass() const { return DataType::mGridClass; }
3829  __hostdev__ bool isLevelSet() const { return DataType::mGridClass == GridClass::LevelSet; }
3830  __hostdev__ bool isFogVolume() const { return DataType::mGridClass == GridClass::FogVolume; }
3831  __hostdev__ bool isStaggered() const { return DataType::mGridClass == GridClass::Staggered; }
3832  __hostdev__ bool isPointIndex() const { return DataType::mGridClass == GridClass::PointIndex; }
3833  __hostdev__ bool isGridIndex() const { return DataType::mGridClass == GridClass::IndexGrid; }
3834  __hostdev__ bool isPointData() const { return DataType::mGridClass == GridClass::PointData; }
3835  __hostdev__ bool isMask() const { return DataType::mGridClass == GridClass::Topology; }
3836  __hostdev__ bool isUnknown() const { return DataType::mGridClass == GridClass::Unknown; }
3837  __hostdev__ bool hasMinMax() const { return DataType::mFlags.isMaskOn(GridFlags::HasMinMax); }
3838  __hostdev__ bool hasBBox() const { return DataType::mFlags.isMaskOn(GridFlags::HasBBox); }
3839  __hostdev__ bool hasLongGridName() const { return DataType::mFlags.isMaskOn(GridFlags::HasLongGridName); }
3840  __hostdev__ bool hasAverage() const { return DataType::mFlags.isMaskOn(GridFlags::HasAverage); }
3841  __hostdev__ bool hasStdDeviation() const { return DataType::mFlags.isMaskOn(GridFlags::HasStdDeviation); }
3842  __hostdev__ bool isBreadthFirst() const { return DataType::mFlags.isMaskOn(GridFlags::IsBreadthFirst); }
3843 
3844  /// @brief return true if the specified node type is layed out breadth-first in memory and has a fixed size.
3845  /// This allows for sequential access to the nodes.
3846  template<typename NodeT>
3847  __hostdev__ bool isSequential() const { return NodeT::FIXED_SIZE && this->isBreadthFirst(); }
3848 
3849  /// @brief return true if the specified node level is layed out breadth-first in memory and has a fixed size.
3850  /// This allows for sequential access to the nodes.
3851  template<int LEVEL>
3852  __hostdev__ bool isSequential() const { return NodeTrait<TreeT, LEVEL>::type::FIXED_SIZE && this->isBreadthFirst(); }
3853 
3854  /// @brief return true if nodes at all levels can safely be accessed with simple linear offsets
3855  __hostdev__ bool isSequential() const { return UpperNodeType::FIXED_SIZE && LowerNodeType::FIXED_SIZE && LeafNodeType::FIXED_SIZE && this->isBreadthFirst(); }
3856 
3857  /// @brief Return a c-string with the name of this grid
3858  __hostdev__ const char* gridName() const { return DataType::gridName(); }
3859 
3860  /// @brief Return a c-string with the name of this grid, truncated to 255 characters
3861  __hostdev__ const char* shortGridName() const { return DataType::mGridName; }
3862 
3863  /// @brief Return checksum of the grid buffer.
3864  __hostdev__ uint64_t checksum() const { return DataType::mChecksum; }
3865 
3866  /// @brief Return true if this grid is empty, i.e. contains no values or nodes.
3867  //__hostdev__ bool isEmpty() const { return this->tree().isEmpty(); }
3868 
3869  /// @brief Return the count of blind-data encoded in this grid
3870  __hostdev__ uint32_t blindDataCount() const { return DataType::mBlindMetadataCount; }
3871 
3872  /// @brief Return the index of the first blind data with specified name if found, otherwise -1.
3873  __hostdev__ int findBlindData(const char* name) const;
3874 
3875  /// @brief Return the index of the first blind data with specified semantic if found, otherwise -1.
3876  __hostdev__ int findBlindDataForSemantic(GridBlindDataSemantic semantic) const;
3877 
3878  /// @brief Returns a const pointer to the blindData at the specified linear offset.
3879  ///
3880  /// @warning Pointer might be NULL and the linear offset is assumed to be in the valid range
3881  // this method is deprecated !!!!
3882  __hostdev__ const void* blindData(uint32_t n) const
3883  {
3884  printf("\nnanovdb::Grid::blindData is unsafe and hence deprecated! Please use nanovdb::Grid::getBlindData instead.\n\n");
3885  NANOVDB_ASSERT(n < DataType::mBlindMetadataCount);
3886  return this->blindMetaData(n).blindData();
3887  }
3888 
3889  template <typename BlindDataT>
3890  __hostdev__ const BlindDataT* getBlindData(uint32_t n) const
3891  {
3892  if (n >= DataType::mBlindMetadataCount) return nullptr;// index is out of bounds
3893  return this->blindMetaData(n).template getBlindData<BlindDataT>();// NULL if mismatching BlindDataT
3894  }
3895 
3896  template <typename BlindDataT>
3897  __hostdev__ BlindDataT* getBlindData(uint32_t n)
3898  {
3899  if (n >= DataType::mBlindMetadataCount) return nullptr;// index is out of bounds
3900  return const_cast<BlindDataT*>(this->blindMetaData(n).template getBlindData<BlindDataT>());// NULL if mismatching BlindDataT
3901  }
3902 
3903  __hostdev__ const GridBlindMetaData& blindMetaData(uint32_t n) const { return *DataType::blindMetaData(n); }
3904 
3905 private:
3906  static_assert(sizeof(GridData) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(GridData) is misaligned");
3907 }; // Class Grid
3908 
3909 template<typename TreeT>
3911 {
3912  for (uint32_t i = 0, n = this->blindDataCount(); i < n; ++i) {
3913  if (this->blindMetaData(i).mSemantic == semantic)
3914  return int(i);
3915  }
3916  return -1;
3917 }
3918 
3919 template<typename TreeT>
3920 __hostdev__ int Grid<TreeT>::findBlindData(const char* name) const
3921 {
3922  auto test = [&](int n) {
3923  const char* str = this->blindMetaData(n).mName;
3924  for (int i = 0; i < GridBlindMetaData::MaxNameSize; ++i) {
3925  if (name[i] != str[i])
3926  return false;
3927  if (name[i] == '\0' && str[i] == '\0')
3928  return true;
3929  }
3930  return true; // all len characters matched
3931  };
3932  for (int i = 0, n = this->blindDataCount(); i < n; ++i)
3933  if (test(i))
3934  return i;
3935  return -1;
3936 }
3937 
3938 // ----------------------------> Tree <--------------------------------------
3939 
3940 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) TreeData
3941 { // sizeof(TreeData) == 64B
3942  int64_t mNodeOffset[4];// 32B, byte offset from this tree to first leaf, lower, upper and root node. A zero offset means no node exists
3943  uint32_t mNodeCount[3]; // 12B, total number of nodes of type: leaf, lower internal, upper internal
3944  uint32_t mTileCount[3]; // 12B, total number of active tile values at the lower internal, upper internal and root node levels
3945  uint64_t mVoxelCount; // 8B, total number of active voxels in the root and all its child nodes.
3946  // No padding since it's always 32B aligned
3948  {
3949  static_assert(8 * 8 == sizeof(TreeData), "TreeData has unexpected size");
3950  memcpy64(this, &other, 8);
3951  return *this;
3952  }
3953  __hostdev__ void setRoot(const void* root) {mNodeOffset[3] = root ? PtrDiff(root, this) : 0;}
3954  __hostdev__ uint8_t* getRoot() { return mNodeOffset[3] ? PtrAdd<uint8_t>(this, mNodeOffset[3]) : nullptr; }
3955  __hostdev__ const uint8_t* getRoot() const { return mNodeOffset[3] ? PtrAdd<uint8_t>(this, mNodeOffset[3]) : nullptr; }
3956 
3957  template<typename NodeT>
3958  __hostdev__ void setFirstNode(const NodeT* node) {mNodeOffset[NodeT::LEVEL] = node ? PtrDiff(node, this) : 0;}
3959 
3960  __hostdev__ bool isEmpty() const {return mNodeOffset[3] ? *PtrAdd<uint32_t>(this, mNodeOffset[3] + sizeof(BBox<Coord>)) == 0 : true;}
3961 
3962  /// @brief Return the index bounding box of all the active values in this tree, i.e. in all nodes of the tree
3963  __hostdev__ CoordBBox bbox() const {return mNodeOffset[3] ? *PtrAdd<CoordBBox>(this, mNodeOffset[3]) : CoordBBox();}
3964 
3965  /// @brief return true if RootData is layout out immediately after TreeData in memory
3966  __hostdev__ bool isRootNext() const {return mNodeOffset[3] ? mNodeOffset[3] == sizeof(TreeData) : false; }
3967 };// TreeData
3968 
3969 // ----------------------------> GridTree <--------------------------------------
3970 
3971 /// @brief defines a tree type from a grid type while preserving constness
3972 template<typename GridT>
3973 struct GridTree
3974 {
3975  using Type = typename GridT::TreeType;
3976  using type = typename GridT::TreeType;
3977 };
3978 template<typename GridT>
3979 struct GridTree<const GridT>
3980 {
3981  using Type = const typename GridT::TreeType;
3982  using type = const typename GridT::TreeType;
3983 };
3984 
3985 // ----------------------------> Tree <--------------------------------------
3986 
3987 /// @brief VDB Tree, which is a thin wrapper around a RootNode.
3988 template<typename RootT>
3989 class Tree : public TreeData
3990 {
3991  static_assert(RootT::LEVEL == 3, "Tree depth is not supported");
3992  static_assert(RootT::ChildNodeType::LOG2DIM == 5, "Tree configuration is not supported");
3993  static_assert(RootT::ChildNodeType::ChildNodeType::LOG2DIM == 4, "Tree configuration is not supported");
3994  static_assert(RootT::LeafNodeType::LOG2DIM == 3, "Tree configuration is not supported");
3995 
3996 public:
3998  using RootType = RootT;
3999  using RootNodeType = RootT;
4000  using UpperNodeType = typename RootNodeType::ChildNodeType;
4001  using LowerNodeType = typename UpperNodeType::ChildNodeType;
4002  using LeafNodeType = typename RootType::LeafNodeType;
4003  using ValueType = typename RootT::ValueType;
4004  using BuildType = typename RootT::BuildType; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
4005  using CoordType = typename RootT::CoordType;
4007 
4008  using Node3 = RootT;
4009  using Node2 = typename RootT::ChildNodeType;
4010  using Node1 = typename Node2::ChildNodeType;
4012 
4013  /// @brief This class cannot be constructed or deleted
4014  Tree() = delete;
4015  Tree(const Tree&) = delete;
4016  Tree& operator=(const Tree&) = delete;
4017  ~Tree() = delete;
4018 
4019  __hostdev__ DataType* data() { return reinterpret_cast<DataType*>(this); }
4020 
4021  __hostdev__ const DataType* data() const { return reinterpret_cast<const DataType*>(this); }
4022 
4023  /// @brief return memory usage in bytes for the class
4024  __hostdev__ static uint64_t memUsage() { return sizeof(DataType); }
4025 
4027  {
4028  RootT* ptr = reinterpret_cast<RootT*>(DataType::getRoot());
4029  NANOVDB_ASSERT(ptr);
4030  return *ptr;
4031  }
4032 
4033  __hostdev__ const RootT& root() const
4034  {
4035  const RootT* ptr = reinterpret_cast<const RootT*>(DataType::getRoot());
4036  NANOVDB_ASSERT(ptr);
4037  return *ptr;
4038  }
4039 
4040  __hostdev__ AccessorType getAccessor() const { return AccessorType(this->root()); }
4041 
4042  /// @brief Return the value of the given voxel (regardless of state or location in the tree.)
4043  __hostdev__ ValueType getValue(const CoordType& ijk) const { return this->root().getValue(ijk); }
4044  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->root().getValue(CoordType(i, j, k)); }
4045 
4046  /// @brief Return the active state of the given voxel (regardless of state or location in the tree.)
4047  __hostdev__ bool isActive(const CoordType& ijk) const { return this->root().isActive(ijk); }
4048 
4049  /// @brief Return true if this tree is empty, i.e. contains no values or nodes
4050  //__hostdev__ bool isEmpty() const { return this->root().isEmpty(); }
4051 
4052  /// @brief Combines the previous two methods in a single call
4053  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->root().probeValue(ijk, v); }
4054 
4055  /// @brief Return a const reference to the background value.
4056  __hostdev__ const ValueType& background() const { return this->root().background(); }
4057 
4058  /// @brief Sets the extrema values of all the active values in this tree, i.e. in all nodes of the tree
4059  __hostdev__ void extrema(ValueType& min, ValueType& max) const;
4060 
4061  /// @brief Return a const reference to the index bounding box of all the active values in this tree, i.e. in all nodes of the tree
4062  //__hostdev__ const BBox<CoordType>& bbox() const { return this->root().bbox(); }
4063 
4064  /// @brief Return the total number of active voxels in this tree.
4065  __hostdev__ uint64_t activeVoxelCount() const { return DataType::mVoxelCount; }
4066 
4067  /// @brief Return the total number of active tiles at the specified level of the tree.
4068  ///
4069  /// @details level = 1,2,3 corresponds to active tile count in lower internal nodes, upper
4070  /// internal nodes, and the root level. Note active values at the leaf level are
4071  /// referred to as active voxels (see activeVoxelCount defined above).
4072  __hostdev__ const uint32_t& activeTileCount(uint32_t level) const
4073  {
4074  NANOVDB_ASSERT(level > 0 && level <= 3); // 1, 2, or 3
4075  return DataType::mTileCount[level - 1];
4076  }
4077 
4078  template<typename NodeT>
4079  __hostdev__ uint32_t nodeCount() const
4080  {
4081  static_assert(NodeT::LEVEL < 3, "Invalid NodeT");
4082  return DataType::mNodeCount[NodeT::LEVEL];
4083  }
4084 
4085  __hostdev__ uint32_t nodeCount(int level) const
4086  {
4087  NANOVDB_ASSERT(level < 3);
4088  return DataType::mNodeCount[level];
4089  }
4090 
4091  __hostdev__ uint32_t totalNodeCount() const
4092  {
4093  return DataType::mNodeCount[0] + DataType::mNodeCount[1] + DataType::mNodeCount[2];
4094  }
4095 
4096  /// @brief return a pointer to the first node of the specified type
4097  ///
4098  /// @warning Note it may return NULL if no nodes exist
4099  template<typename NodeT>
4101  {
4102  const int64_t offset = DataType::mNodeOffset[NodeT::LEVEL];
4103  return offset ? PtrAdd<NodeT>(this, offset) : nullptr;
4104  }
4105 
4106  /// @brief return a const pointer to the first node of the specified type
4107  ///
4108  /// @warning Note it may return NULL if no nodes exist
4109  template<typename NodeT>
4110  __hostdev__ const NodeT* getFirstNode() const
4111  {
4112  const int64_t offset = DataType::mNodeOffset[NodeT::LEVEL];
4113  return offset ? PtrAdd<NodeT>(this, offset) : nullptr;
4114  }
4115 
4116  /// @brief return a pointer to the first node at the specified level
4117  ///
4118  /// @warning Note it may return NULL if no nodes exist
4119  template<int LEVEL>
4122  {
4123  return this->template getFirstNode<typename NodeTrait<RootT, LEVEL>::type>();
4124  }
4125 
4126  /// @brief return a const pointer to the first node of the specified level
4127  ///
4128  /// @warning Note it may return NULL if no nodes exist
4129  template<int LEVEL>
4132  {
4133  return this->template getFirstNode<typename NodeTrait<RootT, LEVEL>::type>();
4134  }
4135 
4136  /// @brief Template specializations of getFirstNode
4137  __hostdev__ LeafNodeType* getFirstLeaf() { return this->getFirstNode<LeafNodeType>(); }
4138  __hostdev__ const LeafNodeType* getFirstLeaf() const { return this->getFirstNode<LeafNodeType>(); }
4139  __hostdev__ typename NodeTrait<RootT, 1>::type* getFirstLower() { return this->getFirstNode<1>(); }
4140  __hostdev__ const typename NodeTrait<RootT, 1>::type* getFirstLower() const { return this->getFirstNode<1>(); }
4141  __hostdev__ typename NodeTrait<RootT, 2>::type* getFirstUpper() { return this->getFirstNode<2>(); }
4142  __hostdev__ const typename NodeTrait<RootT, 2>::type* getFirstUpper() const { return this->getFirstNode<2>(); }
4143 
4144  template<typename OpT, typename... ArgsT>
4145  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
4146  {
4147  return this->root().template get<OpT>(ijk, args...);
4148  }
4149 
4150  template<typename OpT, typename... ArgsT>
4151  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args)
4152  {
4153  return this->root().template set<OpT>(ijk, args...);
4154  }
4155 
4156 private:
4157  static_assert(sizeof(DataType) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(TreeData) is misaligned");
4158 
4159 }; // Tree class
4160 
4161 template<typename RootT>
4163 {
4164  min = this->root().minimum();
4165  max = this->root().maximum();
4166 }
4167 
4168 // --------------------------> RootData <------------------------------------
4169 
4170 /// @brief Struct with all the member data of the RootNode (useful during serialization of an openvdb RootNode)
4171 ///
4172 /// @note No client code should (or can) interface with this struct so it can safely be ignored!
4173 template<typename ChildT>
4174 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) RootData
4175 {
4176  using ValueT = typename ChildT::ValueType;
4177  using BuildT = typename ChildT::BuildType; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
4178  using CoordT = typename ChildT::CoordType;
4179  using StatsT = typename ChildT::FloatType;
4180  static constexpr bool FIXED_SIZE = false;
4181 
4182  /// @brief Return a key based on the coordinates of a voxel
4183 #ifdef NANOVDB_USE_SINGLE_ROOT_KEY
4184  using KeyT = uint64_t;
4185  template<typename CoordType>
4186  __hostdev__ static KeyT CoordToKey(const CoordType& ijk)
4187  {
4188  static_assert(sizeof(CoordT) == sizeof(CoordType), "Mismatching sizeof");
4189  static_assert(32 - ChildT::TOTAL <= 21, "Cannot use 64 bit root keys");
4190  return (KeyT(uint32_t(ijk[2]) >> ChildT::TOTAL)) | // z is the lower 21 bits
4191  (KeyT(uint32_t(ijk[1]) >> ChildT::TOTAL) << 21) | // y is the middle 21 bits
4192  (KeyT(uint32_t(ijk[0]) >> ChildT::TOTAL) << 42); // x is the upper 21 bits
4193  }
4194  __hostdev__ static CoordT KeyToCoord(const KeyT& key)
4195  {
4196  static constexpr uint64_t MASK = (1u << 21) - 1; // used to mask out 21 lower bits
4197  return CoordT(((key >> 42) & MASK) << ChildT::TOTAL, // x are the upper 21 bits
4198  ((key >> 21) & MASK) << ChildT::TOTAL, // y are the middle 21 bits
4199  (key & MASK) << ChildT::TOTAL); // z are the lower 21 bits
4200  }
4201 #else
4202  using KeyT = CoordT;
4203  __hostdev__ static KeyT CoordToKey(const CoordT& ijk) { return ijk & ~ChildT::MASK; }
4204  __hostdev__ static CoordT KeyToCoord(const KeyT& key) { return key; }
4205 #endif
4206  BBox<CoordT> mBBox; // 24B. AABB of active values in index space.
4207  uint32_t mTableSize; // 4B. number of tiles and child pointers in the root node
4208 
4209  ValueT mBackground; // background value, i.e. value of any unset voxel
4210  ValueT mMinimum; // typically 4B, minimum of all the active values
4211  ValueT mMaximum; // typically 4B, maximum of all the active values
4212  StatsT mAverage; // typically 4B, average of all the active values in this node and its child nodes
4213  StatsT mStdDevi; // typically 4B, standard deviation of all the active values in this node and its child nodes
4214 
4215  /// @brief Return padding of this class in bytes, due to aliasing and 32B alignment
4216  ///
4217  /// @note The extra bytes are not necessarily at the end, but can come from aliasing of individual data members.
4218  __hostdev__ static constexpr uint32_t padding()
4219  {
4220  return sizeof(RootData) - (24 + 4 + 3 * sizeof(ValueT) + 2 * sizeof(StatsT));
4221  }
4222 
4223  struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) Tile
4224  {
4225  template<typename CoordType>
4226  __hostdev__ void setChild(const CoordType& k, const void* ptr, const RootData* data)
4227  {
4228  key = CoordToKey(k);
4229  state = false;
4230  child = PtrDiff(ptr, data);
4231  }
4232  template<typename CoordType, typename ValueType>
4233  __hostdev__ void setValue(const CoordType& k, bool s, const ValueType& v)
4234  {
4235  key = CoordToKey(k);
4236  state = s;
4237  value = v;
4238  child = 0;
4239  }
4240  __hostdev__ bool isChild() const { return child != 0; }
4241  __hostdev__ bool isValue() const { return child == 0; }
4242  __hostdev__ bool isActive() const { return child == 0 && state; }
4243  __hostdev__ CoordT origin() const { return KeyToCoord(key); }
4244  KeyT key; // NANOVDB_USE_SINGLE_ROOT_KEY ? 8B : 12B
4245  int64_t child; // 8B. signed byte offset from this node to the child node. 0 means it is a constant tile, so use value.
4246  uint32_t state; // 4B. state of tile value
4247  ValueT value; // value of tile (i.e. no child node)
4248  }; // Tile
4249 
4250  /// @brief Returns a non-const reference to the tile at the specified linear offset.
4251  ///
4252  /// @warning The linear offset is assumed to be in the valid range
4253  __hostdev__ const Tile* tile(uint32_t n) const
4254  {
4255  NANOVDB_ASSERT(n < mTableSize);
4256  return reinterpret_cast<const Tile*>(this + 1) + n;
4257  }
4258  __hostdev__ Tile* tile(uint32_t n)
4259  {
4260  NANOVDB_ASSERT(n < mTableSize);
4261  return reinterpret_cast<Tile*>(this + 1) + n;
4262  }
4263 
4265  {
4266 #if 1 // switch between linear and binary seach
4267  const auto key = CoordToKey(ijk);
4268  for (Tile *p = reinterpret_cast<Tile*>(this + 1), *q = p + mTableSize; p < q; ++p)
4269  if (p->key == key)
4270  return p;
4271  return nullptr;
4272 #else // do not enable binary search if tiles are not guaranteed to be sorted!!!!!!
4273  int32_t low = 0, high = mTableSize; // low is inclusive and high is exclusive
4274  while (low != high) {
4275  int mid = low + ((high - low) >> 1);
4276  const Tile* tile = &tiles[mid];
4277  if (tile->key == key) {
4278  return tile;
4279  } else if (tile->key < key) {
4280  low = mid + 1;
4281  } else {
4282  high = mid;
4283  }
4284  }
4285  return nullptr;
4286 #endif
4287  }
4288 
4289  __hostdev__ inline const Tile* probeTile(const CoordT& ijk) const
4290  {
4291  return const_cast<RootData*>(this)->probeTile(ijk);
4292  }
4293 
4294  /// @brief Returns a const reference to the child node in the specified tile.
4295  ///
4296  /// @warning A child node is assumed to exist in the specified tile
4297  __hostdev__ ChildT* getChild(const Tile* tile)
4298  {
4299  NANOVDB_ASSERT(tile->child);
4300  return PtrAdd<ChildT>(this, tile->child);
4301  }
4302  __hostdev__ const ChildT* getChild(const Tile* tile) const
4303  {
4304  NANOVDB_ASSERT(tile->child);
4305  return PtrAdd<ChildT>(this, tile->child);
4306  }
4307 
4308  __hostdev__ const ValueT& getMin() const { return mMinimum; }
4309  __hostdev__ const ValueT& getMax() const { return mMaximum; }
4310  __hostdev__ const StatsT& average() const { return mAverage; }
4311  __hostdev__ const StatsT& stdDeviation() const { return mStdDevi; }
4312 
4313  __hostdev__ void setMin(const ValueT& v) { mMinimum = v; }
4314  __hostdev__ void setMax(const ValueT& v) { mMaximum = v; }
4315  __hostdev__ void setAvg(const StatsT& v) { mAverage = v; }
4316  __hostdev__ void setDev(const StatsT& v) { mStdDevi = v; }
4317 
4318  /// @brief This class cannot be constructed or deleted
4319  RootData() = delete;
4320  RootData(const RootData&) = delete;
4321  RootData& operator=(const RootData&) = delete;
4322  ~RootData() = delete;
4323 }; // RootData
4324 
4325 // --------------------------> RootNode <------------------------------------
4326 
4327 /// @brief Top-most node of the VDB tree structure.
4328 template<typename ChildT>
4329 class RootNode : public RootData<ChildT>
4330 {
4331 public:
4333  using ChildNodeType = ChildT;
4334  using RootType = RootNode<ChildT>; // this allows RootNode to behave like a Tree
4336  using UpperNodeType = ChildT;
4337  using LowerNodeType = typename UpperNodeType::ChildNodeType;
4338  using LeafNodeType = typename ChildT::LeafNodeType;
4339  using ValueType = typename DataType::ValueT;
4340  using FloatType = typename DataType::StatsT;
4341  using BuildType = typename DataType::BuildT; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
4342 
4343  using CoordType = typename ChildT::CoordType;
4346  using Tile = typename DataType::Tile;
4347  static constexpr bool FIXED_SIZE = DataType::FIXED_SIZE;
4348 
4349  static constexpr uint32_t LEVEL = 1 + ChildT::LEVEL; // level 0 = leaf
4350 
4351  template<typename RootT>
4352  class BaseIter
4353  {
4354  protected:
4358  uint32_t mPos, mSize;
4359  __hostdev__ BaseIter(DataT* data = nullptr, uint32_t n = 0)
4360  : mData(data)
4361  , mPos(0)
4362  , mSize(n)
4363  {
4364  }
4365 
4366  public:
4367  __hostdev__ operator bool() const { return mPos < mSize; }
4368  __hostdev__ uint32_t pos() const { return mPos; }
4369  __hostdev__ void next() { ++mPos; }
4370  __hostdev__ TileT* tile() const { return mData->tile(mPos); }
4372  {
4373  NANOVDB_ASSERT(*this);
4374  return this->tile()->origin();
4375  }
4377  {
4378  NANOVDB_ASSERT(*this);
4379  return this->tile()->origin();
4380  }
4381  }; // Member class BaseIter
4382 
4383  template<typename RootT>
4384  class ChildIter : public BaseIter<RootT>
4385  {
4386  static_assert(is_same<typename remove_const<RootT>::type, RootNode>::value, "Invalid RootT");
4387  using BaseT = BaseIter<RootT>;
4388  using NodeT = typename match_const<ChildT, RootT>::type;
4389 
4390  public:
4392  : BaseT()
4393  {
4394  }
4395  __hostdev__ ChildIter(RootT* parent)
4396  : BaseT(parent->data(), parent->tileCount())
4397  {
4398  NANOVDB_ASSERT(BaseT::mData);
4399  while (*this && !this->tile()->isChild())
4400  this->next();
4401  }
4402  __hostdev__ NodeT& operator*() const
4403  {
4404  NANOVDB_ASSERT(*this);
4405  return *BaseT::mData->getChild(this->tile());
4406  }
4407  __hostdev__ NodeT* operator->() const
4408  {
4409  NANOVDB_ASSERT(*this);
4410  return BaseT::mData->getChild(this->tile());
4411  }
4413  {
4414  NANOVDB_ASSERT(BaseT::mData);
4415  this->next();
4416  while (*this && this->tile()->isValue())
4417  this->next();
4418  return *this;
4419  }
4421  {
4422  auto tmp = *this;
4423  ++(*this);
4424  return tmp;
4425  }
4426  }; // Member class ChildIter
4427 
4430 
4433 
4434  template<typename RootT>
4435  class ValueIter : public BaseIter<RootT>
4436  {
4437  using BaseT = BaseIter<RootT>;
4438 
4439  public:
4441  : BaseT()
4442  {
4443  }
4444  __hostdev__ ValueIter(RootT* parent)
4445  : BaseT(parent->data(), parent->tileCount())
4446  {
4447  NANOVDB_ASSERT(BaseT::mData);
4448  while (*this && this->tile()->isChild())
4449  this->next();
4450  }
4452  {
4453  NANOVDB_ASSERT(*this);
4454  return this->tile()->value;
4455  }
4456  __hostdev__ bool isActive() const
4457  {
4458  NANOVDB_ASSERT(*this);
4459  return this->tile()->state;
4460  }
4462  {
4463  NANOVDB_ASSERT(BaseT::mData);
4464  this->next();
4465  while (*this && this->tile()->isChild())
4466  this->next();
4467  return *this;
4468  }
4470  {
4471  auto tmp = *this;
4472  ++(*this);
4473  return tmp;
4474  }
4475  }; // Member class ValueIter
4476 
4479 
4482 
4483  template<typename RootT>
4484  class ValueOnIter : public BaseIter<RootT>
4485  {
4486  using BaseT = BaseIter<RootT>;
4487 
4488  public:
4490  : BaseT()
4491  {
4492  }
4493  __hostdev__ ValueOnIter(RootT* parent)
4494  : BaseT(parent->data(), parent->tileCount())
4495  {
4496  NANOVDB_ASSERT(BaseT::mData);
4497  while (*this && !this->tile()->isActive())
4498  ++BaseT::mPos;
4499  }
4501  {
4502  NANOVDB_ASSERT(*this);
4503  return this->tile()->value;
4504  }
4506  {
4507  NANOVDB_ASSERT(BaseT::mData);
4508  this->next();
4509  while (*this && !this->tile()->isActive())
4510  this->next();
4511  return *this;
4512  }
4514  {
4515  auto tmp = *this;
4516  ++(*this);
4517  return tmp;
4518  }
4519  }; // Member class ValueOnIter
4520 
4523 
4526 
4527  template<typename RootT>
4528  class DenseIter : public BaseIter<RootT>
4529  {
4530  using BaseT = BaseIter<RootT>;
4531  using NodeT = typename match_const<ChildT, RootT>::type;
4532 
4533  public:
4535  : BaseT()
4536  {
4537  }
4538  __hostdev__ DenseIter(RootT* parent)
4539  : BaseT(parent->data(), parent->tileCount())
4540  {
4541  NANOVDB_ASSERT(BaseT::mData);
4542  }
4543  __hostdev__ NodeT* probeChild(ValueType& value) const
4544  {
4545  NANOVDB_ASSERT(*this);
4546  NodeT* child = nullptr;
4547  auto* t = this->tile();
4548  if (t->isChild()) {
4549  child = BaseT::mData->getChild(t);
4550  } else {
4551  value = t->value;
4552  }
4553  return child;
4554  }
4555  __hostdev__ bool isValueOn() const
4556  {
4557  NANOVDB_ASSERT(*this);
4558  return this->tile()->state;
4559  }
4561  {
4562  NANOVDB_ASSERT(BaseT::mData);
4563  this->next();
4564  return *this;
4565  }
4567  {
4568  auto tmp = *this;
4569  ++(*this);
4570  return tmp;
4571  }
4572  }; // Member class DenseIter
4573 
4576 
4580 
4581  /// @brief This class cannot be constructed or deleted
4582  RootNode() = delete;
4583  RootNode(const RootNode&) = delete;
4584  RootNode& operator=(const RootNode&) = delete;
4585  ~RootNode() = delete;
4586 
4588 
4589  __hostdev__ DataType* data() { return reinterpret_cast<DataType*>(this); }
4590 
4591  __hostdev__ const DataType* data() const { return reinterpret_cast<const DataType*>(this); }
4592 
4593  /// @brief Return a const reference to the index bounding box of all the active values in this tree, i.e. in all nodes of the tree
4594  __hostdev__ const BBoxType& bbox() const { return DataType::mBBox; }
4595 
4596  /// @brief Return the total number of active voxels in the root and all its child nodes.
4597 
4598  /// @brief Return a const reference to the background value, i.e. the value associated with
4599  /// any coordinate location that has not been set explicitly.
4600  __hostdev__ const ValueType& background() const { return DataType::mBackground; }
4601 
4602  /// @brief Return the number of tiles encoded in this root node
4603  __hostdev__ const uint32_t& tileCount() const { return DataType::mTableSize; }
4604  __hostdev__ const uint32_t& getTableSize() const { return DataType::mTableSize; }
4605 
4606  /// @brief Return a const reference to the minimum active value encoded in this root node and any of its child nodes
4607  __hostdev__ const ValueType& minimum() const { return DataType::mMinimum; }
4608 
4609  /// @brief Return a const reference to the maximum active value encoded in this root node and any of its child nodes
4610  __hostdev__ const ValueType& maximum() const { return DataType::mMaximum; }
4611 
4612  /// @brief Return a const reference to the average of all the active values encoded in this root node and any of its child nodes
4613  __hostdev__ const FloatType& average() const { return DataType::mAverage; }
4614 
4615  /// @brief Return the variance of all the active values encoded in this root node and any of its child nodes
4616  __hostdev__ FloatType variance() const { return Pow2(DataType::mStdDevi); }
4617 
4618  /// @brief Return a const reference to the standard deviation of all the active values encoded in this root node and any of its child nodes
4619  __hostdev__ const FloatType& stdDeviation() const { return DataType::mStdDevi; }
4620 
4621  /// @brief Return the expected memory footprint in bytes with the specified number of tiles
4622  __hostdev__ static uint64_t memUsage(uint32_t tableSize) { return sizeof(RootNode) + tableSize * sizeof(Tile); }
4623 
4624  /// @brief Return the actual memory footprint of this root node
4625  __hostdev__ uint64_t memUsage() const { return sizeof(RootNode) + DataType::mTableSize * sizeof(Tile); }
4626 
4627  /// @brief Return true if this RootNode is empty, i.e. contains no values or nodes
4628  __hostdev__ bool isEmpty() const { return DataType::mTableSize == uint32_t(0); }
4629 
4630 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
4631  /// @brief Return the value of the given voxel
4632  __hostdev__ ValueType getValue(const CoordType& ijk) const { return this->template get<GetValue<BuildType>>(ijk); }
4633  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->template get<GetValue<BuildType>>(CoordType(i, j, k)); }
4634  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildType>>(ijk); }
4635  /// @brief return the state and updates the value of the specified voxel
4636  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildType>>(ijk, v); }
4637  __hostdev__ const LeafNodeType* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildType>>(ijk); }
4638 #else // NANOVDB_NEW_ACCESSOR_METHODS
4639 
4640  /// @brief Return the value of the given voxel
4641  __hostdev__ ValueType getValue(const CoordType& ijk) const
4642  {
4643  if (const Tile* tile = DataType::probeTile(ijk)) {
4644  return tile->isChild() ? this->getChild(tile)->getValue(ijk) : tile->value;
4645  }
4646  return DataType::mBackground;
4647  }
4648  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->getValue(CoordType(i, j, k)); }
4649 
4650  __hostdev__ bool isActive(const CoordType& ijk) const
4651  {
4652  if (const Tile* tile = DataType::probeTile(ijk)) {
4653  return tile->isChild() ? this->getChild(tile)->isActive(ijk) : tile->state;
4654  }
4655  return false;
4656  }
4657 
4658  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
4659  {
4660  if (const Tile* tile = DataType::probeTile(ijk)) {
4661  if (tile->isChild()) {
4662  const auto* child = this->getChild(tile);
4663  return child->probeValue(ijk, v);
4664  }
4665  v = tile->value;
4666  return tile->state;
4667  }
4668  v = DataType::mBackground;
4669  return false;
4670  }
4671 
4672  __hostdev__ const LeafNodeType* probeLeaf(const CoordType& ijk) const
4673  {
4674  const Tile* tile = DataType::probeTile(ijk);
4675  if (tile && tile->isChild()) {
4676  const auto* child = this->getChild(tile);
4677  return child->probeLeaf(ijk);
4678  }
4679  return nullptr;
4680  }
4681 
4682 #endif // NANOVDB_NEW_ACCESSOR_METHODS
4683 
4685  {
4686  const Tile* tile = DataType::probeTile(ijk);
4687  return tile && tile->isChild() ? this->getChild(tile) : nullptr;
4688  }
4689 
4691  {
4692  const Tile* tile = DataType::probeTile(ijk);
4693  return tile && tile->isChild() ? this->getChild(tile) : nullptr;
4694  }
4695 
4696  template<typename OpT, typename... ArgsT>
4697  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
4698  {
4699  if (const Tile* tile = this->probeTile(ijk)) {
4700  if (tile->isChild())
4701  return this->getChild(tile)->template get<OpT>(ijk, args...);
4702  return OpT::get(*tile, args...);
4703  }
4704  return OpT::get(*this, args...);
4705  }
4706 
4707  template<typename OpT, typename... ArgsT>
4708  // __hostdev__ auto // occasionally fails with NVCC
4709  __hostdev__ decltype(OpT::set(std::declval<Tile&>(), std::declval<ArgsT>()...))
4710  set(const CoordType& ijk, ArgsT&&... args)
4711  {
4712  if (Tile* tile = DataType::probeTile(ijk)) {
4713  if (tile->isChild())
4714  return this->getChild(tile)->template set<OpT>(ijk, args...);
4715  return OpT::set(*tile, args...);
4716  }
4717  return OpT::set(*this, args...);
4718  }
4719 
4720 private:
4721  static_assert(sizeof(DataType) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(RootData) is misaligned");
4722  static_assert(sizeof(typename DataType::Tile) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(RootData::Tile) is misaligned");
4723 
4724  template<typename, int, int, int>
4725  friend class ReadAccessor;
4726 
4727  template<typename>
4728  friend class Tree;
4729 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
4730  /// @brief Private method to return node information and update a ReadAccessor
4731  template<typename AccT>
4732  __hostdev__ typename AccT::NodeInfo getNodeInfoAndCache(const CoordType& ijk, const AccT& acc) const
4733  {
4734  using NodeInfoT = typename AccT::NodeInfo;
4735  if (const Tile* tile = this->probeTile(ijk)) {
4736  if (tile->isChild()) {
4737  const auto* child = this->getChild(tile);
4738  acc.insert(ijk, child);
4739  return child->getNodeInfoAndCache(ijk, acc);
4740  }
4741  return NodeInfoT{LEVEL, ChildT::dim(), tile->value, tile->value, tile->value, 0, tile->origin(), tile->origin() + CoordType(ChildT::DIM)};
4742  }
4743  return NodeInfoT{LEVEL, ChildT::dim(), this->minimum(), this->maximum(), this->average(), this->stdDeviation(), this->bbox()[0], this->bbox()[1]};
4744  }
4745 
4746  /// @brief Private method to return a voxel value and update a ReadAccessor
4747  template<typename AccT>
4748  __hostdev__ ValueType getValueAndCache(const CoordType& ijk, const AccT& acc) const
4749  {
4750  if (const Tile* tile = this->probeTile(ijk)) {
4751  if (tile->isChild()) {
4752  const auto* child = this->getChild(tile);
4753  acc.insert(ijk, child);
4754  return child->getValueAndCache(ijk, acc);
4755  }
4756  return tile->value;
4757  }
4758  return DataType::mBackground;
4759  }
4760 
4761  template<typename AccT>
4762  __hostdev__ bool isActiveAndCache(const CoordType& ijk, const AccT& acc) const
4763  {
4764  const Tile* tile = this->probeTile(ijk);
4765  if (tile && tile->isChild()) {
4766  const auto* child = this->getChild(tile);
4767  acc.insert(ijk, child);
4768  return child->isActiveAndCache(ijk, acc);
4769  }
4770  return false;
4771  }
4772 
4773  template<typename AccT>
4774  __hostdev__ bool probeValueAndCache(const CoordType& ijk, ValueType& v, const AccT& acc) const
4775  {
4776  if (const Tile* tile = this->probeTile(ijk)) {
4777  if (tile->isChild()) {
4778  const auto* child = this->getChild(tile);
4779  acc.insert(ijk, child);
4780  return child->probeValueAndCache(ijk, v, acc);
4781  }
4782  v = tile->value;
4783  return tile->state;
4784  }
4785  v = DataType::mBackground;
4786  return false;
4787  }
4788 
4789  template<typename AccT>
4790  __hostdev__ const LeafNodeType* probeLeafAndCache(const CoordType& ijk, const AccT& acc) const
4791  {
4792  const Tile* tile = this->probeTile(ijk);
4793  if (tile && tile->isChild()) {
4794  const auto* child = this->getChild(tile);
4795  acc.insert(ijk, child);
4796  return child->probeLeafAndCache(ijk, acc);
4797  }
4798  return nullptr;
4799  }
4800 #endif // NANOVDB_NEW_ACCESSOR_METHODS
4801 
4802  template<typename RayT, typename AccT>
4803  __hostdev__ uint32_t getDimAndCache(const CoordType& ijk, const RayT& ray, const AccT& acc) const
4804  {
4805  if (const Tile* tile = this->probeTile(ijk)) {
4806  if (tile->isChild()) {
4807  const auto* child = this->getChild(tile);
4808  acc.insert(ijk, child);
4809  return child->getDimAndCache(ijk, ray, acc);
4810  }
4811  return 1 << ChildT::TOTAL; //tile value
4812  }
4813  return ChildNodeType::dim(); // background
4814  }
4815 
4816  template<typename OpT, typename AccT, typename... ArgsT>
4817  //__hostdev__ decltype(OpT::get(std::declval<const Tile&>(), std::declval<ArgsT>()...))
4818  __hostdev__ auto
4819  getAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args) const
4820  {
4821  if (const Tile* tile = this->probeTile(ijk)) {
4822  if (tile->isChild()) {
4823  const ChildT* child = this->getChild(tile);
4824  acc.insert(ijk, child);
4825  return child->template getAndCache<OpT>(ijk, acc, args...);
4826  }
4827  return OpT::get(*tile, args...);
4828  }
4829  return OpT::get(*this, args...);
4830  }
4831 
4832  template<typename OpT, typename AccT, typename... ArgsT>
4833  // __hostdev__ auto // occasionally fails with NVCC
4834  __hostdev__ decltype(OpT::set(std::declval<Tile&>(), std::declval<ArgsT>()...))
4835  setAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args)
4836  {
4837  if (Tile* tile = DataType::probeTile(ijk)) {
4838  if (tile->isChild()) {
4839  ChildT* child = this->getChild(tile);
4840  acc.insert(ijk, child);
4841  return child->template setAndCache<OpT>(ijk, acc, args...);
4842  }
4843  return OpT::set(*tile, args...);
4844  }
4845  return OpT::set(*this, args...);
4846  }
4847 
4848 }; // RootNode class
4849 
4850 // After the RootNode the memory layout is assumed to be the sorted Tiles
4851 
4852 // --------------------------> InternalNode <------------------------------------
4853 
4854 /// @brief Struct with all the member data of the InternalNode (useful during serialization of an openvdb InternalNode)
4855 ///
4856 /// @note No client code should (or can) interface with this struct so it can safely be ignored!
4857 template<typename ChildT, uint32_t LOG2DIM>
4858 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) InternalData
4859 {
4860  using ValueT = typename ChildT::ValueType;
4861  using BuildT = typename ChildT::BuildType; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
4862  using StatsT = typename ChildT::FloatType;
4863  using CoordT = typename ChildT::CoordType;
4864  using MaskT = typename ChildT::template MaskType<LOG2DIM>;
4865  static constexpr bool FIXED_SIZE = true;
4866 
4867  union Tile
4868  {
4870  int64_t child; //signed 64 bit byte offset relative to this InternalData, i.e. child-pointer = Tile::child + this
4871  /// @brief This class cannot be constructed or deleted
4872  Tile() = delete;
4873  Tile(const Tile&) = delete;
4874  Tile& operator=(const Tile&) = delete;
4875  ~Tile() = delete;
4876  };
4877 
4878  BBox<CoordT> mBBox; // 24B. node bounding box. |
4879  uint64_t mFlags; // 8B. node flags. | 32B aligned
4880  MaskT mValueMask; // LOG2DIM(5): 4096B, LOG2DIM(4): 512B | 32B aligned
4881  MaskT mChildMask; // LOG2DIM(5): 4096B, LOG2DIM(4): 512B | 32B aligned
4882 
4883  ValueT mMinimum; // typically 4B
4884  ValueT mMaximum; // typically 4B
4885  StatsT mAverage; // typically 4B, average of all the active values in this node and its child nodes
4886  StatsT mStdDevi; // typically 4B, standard deviation of all the active values in this node and its child nodes
4887  // possible padding, e.g. 28 byte padding when ValueType = bool
4888 
4889  /// @brief Return padding of this class in bytes, due to aliasing and 32B alignment
4890  ///
4891  /// @note The extra bytes are not necessarily at the end, but can come from aliasing of individual data members.
4892  __hostdev__ static constexpr uint32_t padding()
4893  {
4894  return sizeof(InternalData) - (24u + 8u + 2 * (sizeof(MaskT) + sizeof(ValueT) + sizeof(StatsT)) + (1u << (3 * LOG2DIM)) * (sizeof(ValueT) > 8u ? sizeof(ValueT) : 8u));
4895  }
4896  alignas(32) Tile mTable[1u << (3 * LOG2DIM)]; // sizeof(ValueT) x (16*16*16 or 32*32*32)
4897 
4898  __hostdev__ static uint64_t memUsage() { return sizeof(InternalData); }
4899 
4900  __hostdev__ void setChild(uint32_t n, const void* ptr)
4901  {
4902  NANOVDB_ASSERT(mChildMask.isOn(n));
4903  mTable[n].child = PtrDiff(ptr, this);
4904  }
4905 
4906  template<typename ValueT>
4907  __hostdev__ void setValue(uint32_t n, const ValueT& v)
4908  {
4909  NANOVDB_ASSERT(!mChildMask.isOn(n));
4910  mTable[n].value = v;
4911  }
4912 
4913  /// @brief Returns a pointer to the child node at the specifed linear offset.
4914  __hostdev__ ChildT* getChild(uint32_t n)
4915  {
4916  NANOVDB_ASSERT(mChildMask.isOn(n));
4917  return PtrAdd<ChildT>(this, mTable[n].child);
4918  }
4919  __hostdev__ const ChildT* getChild(uint32_t n) const
4920  {
4921  NANOVDB_ASSERT(mChildMask.isOn(n));
4922  return PtrAdd<ChildT>(this, mTable[n].child);
4923  }
4924 
4925  __hostdev__ ValueT getValue(uint32_t n) const
4926  {
4927  NANOVDB_ASSERT(mChildMask.isOff(n));
4928  return mTable[n].value;
4929  }
4930 
4931  __hostdev__ bool isActive(uint32_t n) const
4932  {
4933  NANOVDB_ASSERT(mChildMask.isOff(n));
4934  return mValueMask.isOn(n);
4935  }
4936 
4937  __hostdev__ bool isChild(uint32_t n) const { return mChildMask.isOn(n); }
4938 
4939  template<typename T>
4940  __hostdev__ void setOrigin(const T& ijk) { mBBox[0] = ijk; }
4941 
4942  __hostdev__ const ValueT& getMin() const { return mMinimum; }
4943  __hostdev__ const ValueT& getMax() const { return mMaximum; }
4944  __hostdev__ const StatsT& average() const { return mAverage; }
4945  __hostdev__ const StatsT& stdDeviation() const { return mStdDevi; }
4946 
4947 #if defined(__GNUC__) && !defined(__APPLE__) && !defined(__llvm__)
4948 #pragma GCC diagnostic push
4949 #pragma GCC diagnostic ignored "-Wstringop-overflow"
4950 #endif
4951  __hostdev__ void setMin(const ValueT& v) { mMinimum = v; }
4952  __hostdev__ void setMax(const ValueT& v) { mMaximum = v; }
4953  __hostdev__ void setAvg(const StatsT& v) { mAverage = v; }
4954  __hostdev__ void setDev(const StatsT& v) { mStdDevi = v; }
4955 #if defined(__GNUC__) && !defined(__APPLE__) && !defined(__llvm__)
4956 #pragma GCC diagnostic pop
4957 #endif
4958 
4959  /// @brief This class cannot be constructed or deleted
4960  InternalData() = delete;
4961  InternalData(const InternalData&) = delete;
4962  InternalData& operator=(const InternalData&) = delete;
4963  ~InternalData() = delete;
4964 }; // InternalData
4965 
4966 /// @brief Internal nodes of a VDB treedim(),
4967 template<typename ChildT, uint32_t Log2Dim = ChildT::LOG2DIM + 1>
4968 class InternalNode : public InternalData<ChildT, Log2Dim>
4969 {
4970 public:
4972  using ValueType = typename DataType::ValueT;
4973  using FloatType = typename DataType::StatsT;
4974  using BuildType = typename DataType::BuildT; // in rare cases BuildType != ValueType, e.g. then BuildType = ValueMask and ValueType = bool
4975  using LeafNodeType = typename ChildT::LeafNodeType;
4976  using ChildNodeType = ChildT;
4977  using CoordType = typename ChildT::CoordType;
4978  static constexpr bool FIXED_SIZE = DataType::FIXED_SIZE;
4979  template<uint32_t LOG2>
4980  using MaskType = typename ChildT::template MaskType<LOG2>;
4981  template<bool On>
4982  using MaskIterT = typename Mask<Log2Dim>::template Iterator<On>;
4983 
4984  static constexpr uint32_t LOG2DIM = Log2Dim;
4985  static constexpr uint32_t TOTAL = LOG2DIM + ChildT::TOTAL; // dimension in index space
4986  static constexpr uint32_t DIM = 1u << TOTAL; // number of voxels along each axis of this node
4987  static constexpr uint32_t SIZE = 1u << (3 * LOG2DIM); // number of tile values (or child pointers)
4988  static constexpr uint32_t MASK = (1u << TOTAL) - 1u;
4989  static constexpr uint32_t LEVEL = 1 + ChildT::LEVEL; // level 0 = leaf
4990  static constexpr uint64_t NUM_VALUES = uint64_t(1) << (3 * TOTAL); // total voxel count represented by this node
4991 
4992  /// @brief Visits child nodes of this node only
4993  template <typename ParentT>
4994  class ChildIter : public MaskIterT<true>
4995  {
4996  static_assert(is_same<typename remove_const<ParentT>::type, InternalNode>::value, "Invalid ParentT");
4997  using BaseT = MaskIterT<true>;
4998  using NodeT = typename match_const<ChildT, ParentT>::type;
4999  ParentT* mParent;
5000 
5001  public:
5003  : BaseT()
5004  , mParent(nullptr)
5005  {
5006  }
5007  __hostdev__ ChildIter(ParentT* parent)
5008  : BaseT(parent->mChildMask.beginOn())
5009  , mParent(parent)
5010  {
5011  }
5012  ChildIter& operator=(const ChildIter&) = default;
5013  __hostdev__ NodeT& operator*() const
5014  {
5015  NANOVDB_ASSERT(*this);
5016  return *mParent->getChild(BaseT::pos());
5017  }
5018  __hostdev__ NodeT* operator->() const
5019  {
5020  NANOVDB_ASSERT(*this);
5021  return mParent->getChild(BaseT::pos());
5022  }
5024  {
5025  NANOVDB_ASSERT(*this);
5026  return (*this)->origin();
5027  }
5028  __hostdev__ CoordType getCoord() const {return this->getOrigin();}
5029  }; // Member class ChildIter
5030 
5033 
5036 
5037  /// @brief Visits all tile values in this node, i.e. both inactive and active tiles
5038  class ValueIterator : public MaskIterT<false>
5039  {
5040  using BaseT = MaskIterT<false>;
5041  const InternalNode* mParent;
5042 
5043  public:
5045  : BaseT()
5046  , mParent(nullptr)
5047  {
5048  }
5050  : BaseT(parent->data()->mChildMask.beginOff())
5051  , mParent(parent)
5052  {
5053  }
5054  ValueIterator& operator=(const ValueIterator&) = default;
5056  {
5057  NANOVDB_ASSERT(*this);
5058  return mParent->data()->getValue(BaseT::pos());
5059  }
5061  {
5062  NANOVDB_ASSERT(*this);
5063  return mParent->offsetToGlobalCoord(BaseT::pos());
5064  }
5065  __hostdev__ CoordType getCoord() const {return this->getOrigin();}
5066  __hostdev__ bool isActive() const
5067  {
5068  NANOVDB_ASSERT(*this);
5069  return mParent->data()->isActive(BaseT::mPos);
5070  }
5071  }; // Member class ValueIterator
5072 
5075 
5076  /// @brief Visits active tile values of this node only
5077  class ValueOnIterator : public MaskIterT<true>
5078  {
5079  using BaseT = MaskIterT<true>;
5080  const InternalNode* mParent;
5081 
5082  public:
5084  : BaseT()
5085  , mParent(nullptr)
5086  {
5087  }
5089  : BaseT(parent->data()->mValueMask.beginOn())
5090  , mParent(parent)
5091  {
5092  }
5093  ValueOnIterator& operator=(const ValueOnIterator&) = default;
5095  {
5096  NANOVDB_ASSERT(*this);
5097  return mParent->data()->getValue(BaseT::pos());
5098  }
5100  {
5101  NANOVDB_ASSERT(*this);
5102  return mParent->offsetToGlobalCoord(BaseT::pos());
5103  }
5104  __hostdev__ CoordType getCoord() const {return this->getOrigin();}
5105  }; // Member class ValueOnIterator
5106 
5109 
5110  /// @brief Visits all tile values and child nodes of this node
5111  class DenseIterator : public Mask<Log2Dim>::DenseIterator
5112  {
5113  using BaseT = typename Mask<Log2Dim>::DenseIterator;
5114  const DataType* mParent;
5115 
5116  public:
5118  : BaseT()
5119  , mParent(nullptr)
5120  {
5121  }
5123  : BaseT(0)
5124  , mParent(parent->data())
5125  {
5126  }
5127  DenseIterator& operator=(const DenseIterator&) = default;
5128  __hostdev__ const ChildT* probeChild(ValueType& value) const
5129  {
5130  NANOVDB_ASSERT(mParent && bool(*this));
5131  const ChildT* child = nullptr;
5132  if (mParent->mChildMask.isOn(BaseT::pos())) {
5133  child = mParent->getChild(BaseT::pos());
5134  } else {
5135  value = mParent->getValue(BaseT::pos());
5136  }
5137  return child;
5138  }
5139  __hostdev__ bool isValueOn() const
5140  {
5141  NANOVDB_ASSERT(mParent && bool(*this));
5142  return mParent->isActive(BaseT::pos());
5143  }
5145  {
5146  NANOVDB_ASSERT(mParent && bool(*this));
5147  return mParent->offsetToGlobalCoord(BaseT::pos());
5148  }
5149  __hostdev__ CoordType getCoord() const {return this->getOrigin();}
5150  }; // Member class DenseIterator
5151 
5153  __hostdev__ DenseIterator cbeginChildAll() const { return DenseIterator(this); } // matches openvdb
5154 
5155  /// @brief This class cannot be constructed or deleted
5156  InternalNode() = delete;
5157  InternalNode(const InternalNode&) = delete;
5158  InternalNode& operator=(const InternalNode&) = delete;
5159  ~InternalNode() = delete;
5160 
5161  __hostdev__ DataType* data() { return reinterpret_cast<DataType*>(this); }
5162 
5163  __hostdev__ const DataType* data() const { return reinterpret_cast<const DataType*>(this); }
5164 
5165  /// @brief Return the dimension, in voxel units, of this internal node (typically 8*16 or 8*16*32)
5166  __hostdev__ static uint32_t dim() { return 1u << TOTAL; }
5167 
5168  /// @brief Return memory usage in bytes for the class
5169  __hostdev__ static size_t memUsage() { return DataType::memUsage(); }
5170 
5171  /// @brief Return a const reference to the bit mask of active voxels in this internal node
5172  __hostdev__ const MaskType<LOG2DIM>& valueMask() const { return DataType::mValueMask; }
5173  __hostdev__ const MaskType<LOG2DIM>& getValueMask() const { return DataType::mValueMask; }
5174 
5175  /// @brief Return a const reference to the bit mask of child nodes in this internal node
5176  __hostdev__ const MaskType<LOG2DIM>& childMask() const { return DataType::mChildMask; }
5177  __hostdev__ const MaskType<LOG2DIM>& getChildMask() const { return DataType::mChildMask; }
5178 
5179  /// @brief Return the origin in index space of this leaf node
5180  __hostdev__ CoordType origin() const { return DataType::mBBox.min() & ~MASK; }
5181 
5182  /// @brief Return a const reference to the minimum active value encoded in this internal node and any of its child nodes
5183  __hostdev__ const ValueType& minimum() const { return this->getMin(); }
5184 
5185  /// @brief Return a const reference to the maximum active value encoded in this internal node and any of its child nodes
5186  __hostdev__ const ValueType& maximum() const { return this->getMax(); }
5187 
5188  /// @brief Return a const reference to the average of all the active values encoded in this internal node and any of its child nodes
5189  __hostdev__ const FloatType& average() const { return DataType::mAverage; }
5190 
5191  /// @brief Return the variance of all the active values encoded in this internal node and any of its child nodes
5192  __hostdev__ FloatType variance() const { return DataType::mStdDevi * DataType::mStdDevi; }
5193 
5194  /// @brief Return a const reference to the standard deviation of all the active values encoded in this internal node and any of its child nodes
5195  __hostdev__ const FloatType& stdDeviation() const { return DataType::mStdDevi; }
5196 
5197  /// @brief Return a const reference to the bounding box in index space of active values in this internal node and any of its child nodes
5198  __hostdev__ const BBox<CoordType>& bbox() const { return DataType::mBBox; }
5199 
5200  /// @brief If the first entry in this node's table is a tile, return the tile's value.
5201  /// Otherwise, return the result of calling getFirstValue() on the child.
5203  {
5204  return DataType::mChildMask.isOn(0) ? this->getChild(0)->getFirstValue() : DataType::getValue(0);
5205  }
5206 
5207  /// @brief If the last entry in this node's table is a tile, return the tile's value.
5208  /// Otherwise, return the result of calling getLastValue() on the child.
5210  {
5211  return DataType::mChildMask.isOn(SIZE - 1) ? this->getChild(SIZE - 1)->getLastValue() : DataType::getValue(SIZE - 1);
5212  }
5213 
5214 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
5215  /// @brief Return the value of the given voxel
5216  __hostdev__ ValueType getValue(const CoordType& ijk) const { return this->template get<GetValue<BuildType>>(ijk); }
5217  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildType>>(ijk); }
5218  /// @brief return the state and updates the value of the specified voxel
5219  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildType>>(ijk, v); }
5220  __hostdev__ const LeafNodeType* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildType>>(ijk); }
5221 #else // NANOVDB_NEW_ACCESSOR_METHODS
5222  __hostdev__ ValueType getValue(const CoordType& ijk) const
5223  {
5224  const uint32_t n = CoordToOffset(ijk);
5225  return DataType::mChildMask.isOn(n) ? this->getChild(n)->getValue(ijk) : DataType::getValue(n);
5226  }
5227  __hostdev__ bool isActive(const CoordType& ijk) const
5228  {
5229  const uint32_t n = CoordToOffset(ijk);
5230  return DataType::mChildMask.isOn(n) ? this->getChild(n)->isActive(ijk) : DataType::isActive(n);
5231  }
5232  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
5233  {
5234  const uint32_t n = CoordToOffset(ijk);
5235  if (DataType::mChildMask.isOn(n))
5236  return this->getChild(n)->probeValue(ijk, v);
5237  v = DataType::getValue(n);
5238  return DataType::isActive(n);
5239  }
5240  __hostdev__ const LeafNodeType* probeLeaf(const CoordType& ijk) const
5241  {
5242  const uint32_t n = CoordToOffset(ijk);
5243  if (DataType::mChildMask.isOn(n))
5244  return this->getChild(n)->probeLeaf(ijk);
5245  return nullptr;
5246  }
5247 
5248 #endif // NANOVDB_NEW_ACCESSOR_METHODS
5249 
5251  {
5252  const uint32_t n = CoordToOffset(ijk);
5253  return DataType::mChildMask.isOn(n) ? this->getChild(n) : nullptr;
5254  }
5256  {
5257  const uint32_t n = CoordToOffset(ijk);
5258  return DataType::mChildMask.isOn(n) ? this->getChild(n) : nullptr;
5259  }
5260 
5261  /// @brief Return the linear offset corresponding to the given coordinate
5262  __hostdev__ static uint32_t CoordToOffset(const CoordType& ijk)
5263  {
5264  return (((ijk[0] & MASK) >> ChildT::TOTAL) << (2 * LOG2DIM)) | // note, we're using bitwise OR instead of +
5265  (((ijk[1] & MASK) >> ChildT::TOTAL) << (LOG2DIM)) |
5266  ((ijk[2] & MASK) >> ChildT::TOTAL);
5267  }
5268 
5269  /// @return the local coordinate of the n'th tile or child node
5271  {
5272  NANOVDB_ASSERT(n < SIZE);
5273  const uint32_t m = n & ((1 << 2 * LOG2DIM) - 1);
5274  return Coord(n >> 2 * LOG2DIM, m >> LOG2DIM, m & ((1 << LOG2DIM) - 1));
5275  }
5276 
5277  /// @brief modifies local coordinates to global coordinates of a tile or child node
5279  {
5280  ijk <<= ChildT::TOTAL;
5281  ijk += this->origin();
5282  }
5283 
5285  {
5287  this->localToGlobalCoord(ijk);
5288  return ijk;
5289  }
5290 
5291  /// @brief Return true if this node or any of its child nodes contain active values
5292  __hostdev__ bool isActive() const { return DataType::mFlags & uint32_t(2); }
5293 
5294  template<typename OpT, typename... ArgsT>
5295  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
5296  {
5297  const uint32_t n = CoordToOffset(ijk);
5298  if (this->isChild(n))
5299  return this->getChild(n)->template get<OpT>(ijk, args...);
5300  return OpT::get(*this, n, args...);
5301  }
5302 
5303  template<typename OpT, typename... ArgsT>
5304  //__hostdev__ auto // occasionally fails with NVCC
5305  __hostdev__ decltype(OpT::set(std::declval<InternalNode&>(), std::declval<uint32_t>(), std::declval<ArgsT>()...))
5306  set(const CoordType& ijk, ArgsT&&... args)
5307  {
5308  const uint32_t n = CoordToOffset(ijk);
5309  if (this->isChild(n))
5310  return this->getChild(n)->template set<OpT>(ijk, args...);
5311  return OpT::set(*this, n, args...);
5312  }
5313 
5314 private:
5315  static_assert(sizeof(DataType) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(InternalData) is misaligned");
5316 
5317  template<typename, int, int, int>
5318  friend class ReadAccessor;
5319 
5320  template<typename>
5321  friend class RootNode;
5322  template<typename, uint32_t>
5323  friend class InternalNode;
5324 
5325 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
5326  /// @brief Private read access method used by the ReadAccessor
5327  template<typename AccT>
5328  __hostdev__ ValueType getValueAndCache(const CoordType& ijk, const AccT& acc) const
5329  {
5330  const uint32_t n = CoordToOffset(ijk);
5331  if (DataType::mChildMask.isOff(n))
5332  return DataType::getValue(n);
5333  const ChildT* child = this->getChild(n);
5334  acc.insert(ijk, child);
5335  return child->getValueAndCache(ijk, acc);
5336  }
5337  template<typename AccT>
5338  __hostdev__ bool isActiveAndCache(const CoordType& ijk, const AccT& acc) const
5339  {
5340  const uint32_t n = CoordToOffset(ijk);
5341  if (DataType::mChildMask.isOff(n))
5342  return DataType::isActive(n);
5343  const ChildT* child = this->getChild(n);
5344  acc.insert(ijk, child);
5345  return child->isActiveAndCache(ijk, acc);
5346  }
5347  template<typename AccT>
5348  __hostdev__ bool probeValueAndCache(const CoordType& ijk, ValueType& v, const AccT& acc) const
5349  {
5350  const uint32_t n = CoordToOffset(ijk);
5351  if (DataType::mChildMask.isOff(n)) {
5352  v = DataType::getValue(n);
5353  return DataType::isActive(n);
5354  }
5355  const ChildT* child = this->getChild(n);
5356  acc.insert(ijk, child);
5357  return child->probeValueAndCache(ijk, v, acc);
5358  }
5359  template<typename AccT>
5360  __hostdev__ const LeafNodeType* probeLeafAndCache(const CoordType& ijk, const AccT& acc) const
5361  {
5362  const uint32_t n = CoordToOffset(ijk);
5363  if (DataType::mChildMask.isOff(n))
5364  return nullptr;
5365  const ChildT* child = this->getChild(n);
5366  acc.insert(ijk, child);
5367  return child->probeLeafAndCache(ijk, acc);
5368  }
5369  template<typename AccT>
5370  __hostdev__ typename AccT::NodeInfo getNodeInfoAndCache(const CoordType& ijk, const AccT& acc) const
5371  {
5372  using NodeInfoT = typename AccT::NodeInfo;
5373  const uint32_t n = CoordToOffset(ijk);
5374  if (DataType::mChildMask.isOff(n)) {
5375  return NodeInfoT{LEVEL, this->dim(), this->minimum(), this->maximum(), this->average(), this->stdDeviation(), this->bbox()[0], this->bbox()[1]};
5376  }
5377  const ChildT* child = this->getChild(n);
5378  acc.insert(ijk, child);
5379  return child->getNodeInfoAndCache(ijk, acc);
5380  }
5381 #endif // NANOVDB_NEW_ACCESSOR_METHODS
5382 
5383  template<typename RayT, typename AccT>
5384  __hostdev__ uint32_t getDimAndCache(const CoordType& ijk, const RayT& ray, const AccT& acc) const
5385  {
5386  if (DataType::mFlags & uint32_t(1u))
5387  return this->dim(); // skip this node if the 1st bit is set
5388  //if (!ray.intersects( this->bbox() )) return 1<<TOTAL;
5389 
5390  const uint32_t n = CoordToOffset(ijk);
5391  if (DataType::mChildMask.isOn(n)) {
5392  const ChildT* child = this->getChild(n);
5393  acc.insert(ijk, child);
5394  return child->getDimAndCache(ijk, ray, acc);
5395  }
5396  return ChildNodeType::dim(); // tile value
5397  }
5398 
5399  template<typename OpT, typename AccT, typename... ArgsT>
5400  __hostdev__ auto
5401  //__hostdev__ decltype(OpT::get(std::declval<const InternalNode&>(), std::declval<uint32_t>(), std::declval<ArgsT>()...))
5402  getAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args) const
5403  {
5404  const uint32_t n = CoordToOffset(ijk);
5405  if (DataType::mChildMask.isOff(n))
5406  return OpT::get(*this, n, args...);
5407  const ChildT* child = this->getChild(n);
5408  acc.insert(ijk, child);
5409  return child->template getAndCache<OpT>(ijk, acc, args...);
5410  }
5411 
5412  template<typename OpT, typename AccT, typename... ArgsT>
5413  //__hostdev__ auto // occasionally fails with NVCC
5414  __hostdev__ decltype(OpT::set(std::declval<InternalNode&>(), std::declval<uint32_t>(), std::declval<ArgsT>()...))
5415  setAndCache(const CoordType& ijk, const AccT& acc, ArgsT&&... args)
5416  {
5417  const uint32_t n = CoordToOffset(ijk);
5418  if (DataType::mChildMask.isOff(n))
5419  return OpT::set(*this, n, args...);
5420  ChildT* child = this->getChild(n);
5421  acc.insert(ijk, child);
5422  return child->template setAndCache<OpT>(ijk, acc, args...);
5423  }
5424 
5425 }; // InternalNode class
5426 
5427 // --------------------------> LeafData<T> <------------------------------------
5428 
5429 /// @brief Stuct with all the member data of the LeafNode (useful during serialization of an openvdb LeafNode)
5430 ///
5431 /// @note No client code should (or can) interface with this struct so it can safely be ignored!
5432 template<typename ValueT, typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5433 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData
5434 {
5435  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5436  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5437  using ValueType = ValueT;
5438  using BuildType = ValueT;
5440  using ArrayType = ValueT; // type used for the internal mValue array
5441  static constexpr bool FIXED_SIZE = true;
5442 
5443  CoordT mBBoxMin; // 12B.
5444  uint8_t mBBoxDif[3]; // 3B.
5445  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5446  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5447 
5448  ValueType mMinimum; // typically 4B
5449  ValueType mMaximum; // typically 4B
5450  FloatType mAverage; // typically 4B, average of all the active values in this node and its child nodes
5451  FloatType mStdDevi; // typically 4B, standard deviation of all the active values in this node and its child nodes
5452  alignas(32) ValueType mValues[1u << 3 * LOG2DIM];
5453 
5454  /// @brief Return padding of this class in bytes, due to aliasing and 32B alignment
5455  ///
5456  /// @note The extra bytes are not necessarily at the end, but can come from aliasing of individual data members.
5457  __hostdev__ static constexpr uint32_t padding()
5458  {
5459  return sizeof(LeafData) - (12 + 3 + 1 + sizeof(MaskT<LOG2DIM>) + 2 * (sizeof(ValueT) + sizeof(FloatType)) + (1u << (3 * LOG2DIM)) * sizeof(ValueT));
5460  }
5461  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5462 
5463  __hostdev__ static bool hasStats() { return true; }
5464 
5465  __hostdev__ ValueType getValue(uint32_t i) const { return mValues[i]; }
5466  __hostdev__ void setValueOnly(uint32_t offset, const ValueType& value) { mValues[offset] = value; }
5467  __hostdev__ void setValue(uint32_t offset, const ValueType& value)
5468  {
5469  mValueMask.setOn(offset);
5470  mValues[offset] = value;
5471  }
5472  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5473 
5474  __hostdev__ ValueType getMin() const { return mMinimum; }
5475  __hostdev__ ValueType getMax() const { return mMaximum; }
5476  __hostdev__ FloatType getAvg() const { return mAverage; }
5477  __hostdev__ FloatType getDev() const { return mStdDevi; }
5478 
5479  __hostdev__ void setMin(const ValueType& v) { mMinimum = v; }
5480  __hostdev__ void setMax(const ValueType& v) { mMaximum = v; }
5481  __hostdev__ void setAvg(const FloatType& v) { mAverage = v; }
5482  __hostdev__ void setDev(const FloatType& v) { mStdDevi = v; }
5483 
5484  template<typename T>
5485  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
5486 
5487  __hostdev__ void fill(const ValueType& v)
5488  {
5489  for (auto *p = mValues, *q = p + 512; p != q; ++p)
5490  *p = v;
5491  }
5492 
5493  /// @brief This class cannot be constructed or deleted
5494  LeafData() = delete;
5495  LeafData(const LeafData&) = delete;
5496  LeafData& operator=(const LeafData&) = delete;
5497  ~LeafData() = delete;
5498 }; // LeafData<ValueT>
5499 
5500 // --------------------------> LeafFnBase <------------------------------------
5501 
5502 /// @brief Base-class for quantized float leaf nodes
5503 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5504 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafFnBase
5505 {
5506  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5507  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5508  using ValueType = float;
5509  using FloatType = float;
5510 
5511  CoordT mBBoxMin; // 12B.
5512  uint8_t mBBoxDif[3]; // 3B.
5513  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5514  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5515 
5516  float mMinimum; // 4B - minimum of ALL values in this node
5517  float mQuantum; // = (max - min)/15 4B
5518  uint16_t mMin, mMax, mAvg, mDev; // quantized representations of statistics of active values
5519  // no padding since it's always 32B aligned
5520  __hostdev__ static uint64_t memUsage() { return sizeof(LeafFnBase); }
5521 
5522  __hostdev__ static bool hasStats() { return true; }
5523 
5524  /// @brief Return padding of this class in bytes, due to aliasing and 32B alignment
5525  ///
5526  /// @note The extra bytes are not necessarily at the end, but can come from aliasing of individual data members.
5527  __hostdev__ static constexpr uint32_t padding()
5528  {
5529  return sizeof(LeafFnBase) - (12 + 3 + 1 + sizeof(MaskT<LOG2DIM>) + 2 * 4 + 4 * 2);
5530  }
5531  __hostdev__ void init(float min, float max, uint8_t bitWidth)
5532  {
5533  mMinimum = min;
5534  mQuantum = (max - min) / float((1 << bitWidth) - 1);
5535  }
5536 
5537  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5538 
5539  /// @brief return the quantized minimum of the active values in this node
5540  __hostdev__ float getMin() const { return mMin * mQuantum + mMinimum; }
5541 
5542  /// @brief return the quantized maximum of the active values in this node
5543  __hostdev__ float getMax() const { return mMax * mQuantum + mMinimum; }
5544 
5545  /// @brief return the quantized average of the active values in this node
5546  __hostdev__ float getAvg() const { return mAvg * mQuantum + mMinimum; }
5547  /// @brief return the quantized standard deviation of the active values in this node
5548 
5549  /// @note 0 <= StdDev <= max-min or 0 <= StdDev/(max-min) <= 1
5550  __hostdev__ float getDev() const { return mDev * mQuantum; }
5551 
5552  /// @note min <= X <= max or 0 <= (X-min)/(min-max) <= 1
5553  __hostdev__ void setMin(float min) { mMin = uint16_t((min - mMinimum) / mQuantum + 0.5f); }
5554 
5555  /// @note min <= X <= max or 0 <= (X-min)/(min-max) <= 1
5556  __hostdev__ void setMax(float max) { mMax = uint16_t((max - mMinimum) / mQuantum + 0.5f); }
5557 
5558  /// @note min <= avg <= max or 0 <= (avg-min)/(min-max) <= 1
5559  __hostdev__ void setAvg(float avg) { mAvg = uint16_t((avg - mMinimum) / mQuantum + 0.5f); }
5560 
5561  /// @note 0 <= StdDev <= max-min or 0 <= StdDev/(max-min) <= 1
5562  __hostdev__ void setDev(float dev) { mDev = uint16_t(dev / mQuantum + 0.5f); }
5563 
5564  template<typename T>
5565  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
5566 }; // LeafFnBase
5567 
5568 // --------------------------> LeafData<Fp4> <------------------------------------
5569 
5570 /// @brief Stuct with all the member data of the LeafNode (useful during serialization of an openvdb LeafNode)
5571 ///
5572 /// @note No client code should (or can) interface with this struct so it can safely be ignored!
5573 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5574 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<Fp4, CoordT, MaskT, LOG2DIM>
5575  : public LeafFnBase<CoordT, MaskT, LOG2DIM>
5576 {
5578  using BuildType = Fp4;
5579  using ArrayType = uint8_t; // type used for the internal mValue array
5580  static constexpr bool FIXED_SIZE = true;
5581  alignas(32) uint8_t mCode[1u << (3 * LOG2DIM - 1)]; // LeafFnBase is 32B aligned and so is mCode
5582 
5583  __hostdev__ static constexpr uint64_t memUsage() { return sizeof(LeafData); }
5584  __hostdev__ static constexpr uint32_t padding()
5585  {
5586  static_assert(BaseT::padding() == 0, "expected no padding in LeafFnBase");
5587  return sizeof(LeafData) - sizeof(BaseT) - (1u << (3 * LOG2DIM - 1));
5588  }
5589 
5590  __hostdev__ static constexpr uint8_t bitWidth() { return 4u; }
5591  __hostdev__ float getValue(uint32_t i) const
5592  {
5593 #if 0
5594  const uint8_t c = mCode[i>>1];
5595  return ( (i&1) ? c >> 4 : c & uint8_t(15) )*BaseT::mQuantum + BaseT::mMinimum;
5596 #else
5597  return ((mCode[i >> 1] >> ((i & 1) << 2)) & uint8_t(15)) * BaseT::mQuantum + BaseT::mMinimum;
5598 #endif
5599  }
5600 
5601  /// @brief This class cannot be constructed or deleted
5602  LeafData() = delete;
5603  LeafData(const LeafData&) = delete;
5604  LeafData& operator=(const LeafData&) = delete;
5605  ~LeafData() = delete;
5606 }; // LeafData<Fp4>
5607 
5608 // --------------------------> LeafBase<Fp8> <------------------------------------
5609 
5610 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5611 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<Fp8, CoordT, MaskT, LOG2DIM>
5612  : public LeafFnBase<CoordT, MaskT, LOG2DIM>
5613 {
5615  using BuildType = Fp8;
5616  using ArrayType = uint8_t; // type used for the internal mValue array
5617  static constexpr bool FIXED_SIZE = true;
5618  alignas(32) uint8_t mCode[1u << 3 * LOG2DIM];
5619  __hostdev__ static constexpr int64_t memUsage() { return sizeof(LeafData); }
5620  __hostdev__ static constexpr uint32_t padding()
5621  {
5622  static_assert(BaseT::padding() == 0, "expected no padding in LeafFnBase");
5623  return sizeof(LeafData) - sizeof(BaseT) - (1u << 3 * LOG2DIM);
5624  }
5625 
5626  __hostdev__ static constexpr uint8_t bitWidth() { return 8u; }
5627  __hostdev__ float getValue(uint32_t i) const
5628  {
5629  return mCode[i] * BaseT::mQuantum + BaseT::mMinimum; // code * (max-min)/255 + min
5630  }
5631  /// @brief This class cannot be constructed or deleted
5632  LeafData() = delete;
5633  LeafData(const LeafData&) = delete;
5634  LeafData& operator=(const LeafData&) = delete;
5635  ~LeafData() = delete;
5636 }; // LeafData<Fp8>
5637 
5638 // --------------------------> LeafData<Fp16> <------------------------------------
5639 
5640 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5641 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<Fp16, CoordT, MaskT, LOG2DIM>
5642  : public LeafFnBase<CoordT, MaskT, LOG2DIM>
5643 {
5645  using BuildType = Fp16;
5646  using ArrayType = uint16_t; // type used for the internal mValue array
5647  static constexpr bool FIXED_SIZE = true;
5648  alignas(32) uint16_t mCode[1u << 3 * LOG2DIM];
5649 
5650  __hostdev__ static constexpr uint64_t memUsage() { return sizeof(LeafData); }
5651  __hostdev__ static constexpr uint32_t padding()
5652  {
5653  static_assert(BaseT::padding() == 0, "expected no padding in LeafFnBase");
5654  return sizeof(LeafData) - sizeof(BaseT) - 2 * (1u << 3 * LOG2DIM);
5655  }
5656 
5657  __hostdev__ static constexpr uint8_t bitWidth() { return 16u; }
5658  __hostdev__ float getValue(uint32_t i) const
5659  {
5660  return mCode[i] * BaseT::mQuantum + BaseT::mMinimum; // code * (max-min)/65535 + min
5661  }
5662 
5663  /// @brief This class cannot be constructed or deleted
5664  LeafData() = delete;
5665  LeafData(const LeafData&) = delete;
5666  LeafData& operator=(const LeafData&) = delete;
5667  ~LeafData() = delete;
5668 }; // LeafData<Fp16>
5669 
5670 // --------------------------> LeafData<FpN> <------------------------------------
5671 
5672 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5673 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<FpN, CoordT, MaskT, LOG2DIM>
5674  : public LeafFnBase<CoordT, MaskT, LOG2DIM>
5675 { // this class has no additional data members, however every instance is immediately followed by
5676  // bitWidth*64 bytes. Since its base class is 32B aligned so are the bitWidth*64 bytes
5678  using BuildType = FpN;
5679  static constexpr bool FIXED_SIZE = false;
5680  __hostdev__ static constexpr uint32_t padding()
5681  {
5682  static_assert(BaseT::padding() == 0, "expected no padding in LeafFnBase");
5683  return 0;
5684  }
5685 
5686  __hostdev__ uint8_t bitWidth() const { return 1 << (BaseT::mFlags >> 5); } // 4,8,16,32 = 2^(2,3,4,5)
5687  __hostdev__ size_t memUsage() const { return sizeof(*this) + this->bitWidth() * 64; }
5688  __hostdev__ static size_t memUsage(uint32_t bitWidth) { return 96u + bitWidth * 64; }
5689  __hostdev__ float getValue(uint32_t i) const
5690  {
5691 #ifdef NANOVDB_FPN_BRANCHLESS // faster
5692  const int b = BaseT::mFlags >> 5; // b = 0, 1, 2, 3, 4 corresponding to 1, 2, 4, 8, 16 bits
5693 #if 0 // use LUT
5694  uint16_t code = reinterpret_cast<const uint16_t*>(this + 1)[i >> (4 - b)];
5695  const static uint8_t shift[5] = {15, 7, 3, 1, 0};
5696  const static uint16_t mask[5] = {1, 3, 15, 255, 65535};
5697  code >>= (i & shift[b]) << b;
5698  code &= mask[b];
5699 #else // no LUT
5700  uint32_t code = reinterpret_cast<const uint32_t*>(this + 1)[i >> (5 - b)];
5701  code >>= (i & ((32 >> b) - 1)) << b;
5702  code &= (1 << (1 << b)) - 1;
5703 #endif
5704 #else // use branched version (slow)
5705  float code;
5706  auto* values = reinterpret_cast<const uint8_t*>(this + 1);
5707  switch (BaseT::mFlags >> 5) {
5708  case 0u: // 1 bit float
5709  code = float((values[i >> 3] >> (i & 7)) & uint8_t(1));
5710  break;
5711  case 1u: // 2 bits float
5712  code = float((values[i >> 2] >> ((i & 3) << 1)) & uint8_t(3));
5713  break;
5714  case 2u: // 4 bits float
5715  code = float((values[i >> 1] >> ((i & 1) << 2)) & uint8_t(15));
5716  break;
5717  case 3u: // 8 bits float
5718  code = float(values[i]);
5719  break;
5720  default: // 16 bits float
5721  code = float(reinterpret_cast<const uint16_t*>(values)[i]);
5722  }
5723 #endif
5724  return float(code) * BaseT::mQuantum + BaseT::mMinimum; // code * (max-min)/UNITS + min
5725  }
5726 
5727  /// @brief This class cannot be constructed or deleted
5728  LeafData() = delete;
5729  LeafData(const LeafData&) = delete;
5730  LeafData& operator=(const LeafData&) = delete;
5731  ~LeafData() = delete;
5732 }; // LeafData<FpN>
5733 
5734 // --------------------------> LeafData<bool> <------------------------------------
5735 
5736 // Partial template specialization of LeafData with bool
5737 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5738 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<bool, CoordT, MaskT, LOG2DIM>
5739 {
5740  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5741  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5742  using ValueType = bool;
5743  using BuildType = bool;
5744  using FloatType = bool; // dummy value type
5745  using ArrayType = MaskT<LOG2DIM>; // type used for the internal mValue array
5746  static constexpr bool FIXED_SIZE = true;
5747 
5748  CoordT mBBoxMin; // 12B.
5749  uint8_t mBBoxDif[3]; // 3B.
5750  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5751  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5752  MaskT<LOG2DIM> mValues; // LOG2DIM(3): 64B.
5753  uint64_t mPadding[2]; // 16B padding to 32B alignment
5754 
5755  __hostdev__ static constexpr uint32_t padding() { return sizeof(LeafData) - 12u - 3u - 1u - 2 * sizeof(MaskT<LOG2DIM>) - 16u; }
5756  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5757  __hostdev__ static bool hasStats() { return false; }
5758  __hostdev__ bool getValue(uint32_t i) const { return mValues.isOn(i); }
5759  __hostdev__ bool getMin() const { return false; } // dummy
5760  __hostdev__ bool getMax() const { return false; } // dummy
5761  __hostdev__ bool getAvg() const { return false; } // dummy
5762  __hostdev__ bool getDev() const { return false; } // dummy
5763  __hostdev__ void setValue(uint32_t offset, bool v)
5764  {
5765  mValueMask.setOn(offset);
5766  mValues.set(offset, v);
5767  }
5768  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5769  __hostdev__ void setMin(const bool&) {} // no-op
5770  __hostdev__ void setMax(const bool&) {} // no-op
5771  __hostdev__ void setAvg(const bool&) {} // no-op
5772  __hostdev__ void setDev(const bool&) {} // no-op
5773 
5774  template<typename T>
5775  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
5776 
5777  /// @brief This class cannot be constructed or deleted
5778  LeafData() = delete;
5779  LeafData(const LeafData&) = delete;
5780  LeafData& operator=(const LeafData&) = delete;
5781  ~LeafData() = delete;
5782 }; // LeafData<bool>
5783 
5784 // --------------------------> LeafData<ValueMask> <------------------------------------
5785 
5786 // Partial template specialization of LeafData with ValueMask
5787 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5788 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<ValueMask, CoordT, MaskT, LOG2DIM>
5789 {
5790  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5791  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5792  using ValueType = bool;
5794  using FloatType = bool; // dummy value type
5795  using ArrayType = void; // type used for the internal mValue array - void means missing
5796  static constexpr bool FIXED_SIZE = true;
5797 
5798  CoordT mBBoxMin; // 12B.
5799  uint8_t mBBoxDif[3]; // 3B.
5800  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5801  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5802  uint64_t mPadding[2]; // 16B padding to 32B alignment
5803 
5804  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5805  __hostdev__ static bool hasStats() { return false; }
5806  __hostdev__ static constexpr uint32_t padding()
5807  {
5808  return sizeof(LeafData) - (12u + 3u + 1u + sizeof(MaskT<LOG2DIM>) + 2 * 8u);
5809  }
5810 
5811  __hostdev__ bool getValue(uint32_t i) const { return mValueMask.isOn(i); }
5812  __hostdev__ bool getMin() const { return false; } // dummy
5813  __hostdev__ bool getMax() const { return false; } // dummy
5814  __hostdev__ bool getAvg() const { return false; } // dummy
5815  __hostdev__ bool getDev() const { return false; } // dummy
5816  __hostdev__ void setValue(uint32_t offset, bool) { mValueMask.setOn(offset); }
5817  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5818  __hostdev__ void setMin(const ValueType&) {} // no-op
5819  __hostdev__ void setMax(const ValueType&) {} // no-op
5820  __hostdev__ void setAvg(const FloatType&) {} // no-op
5821  __hostdev__ void setDev(const FloatType&) {} // no-op
5822 
5823  template<typename T>
5824  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
5825 
5826  /// @brief This class cannot be constructed or deleted
5827  LeafData() = delete;
5828  LeafData(const LeafData&) = delete;
5829  LeafData& operator=(const LeafData&) = delete;
5830  ~LeafData() = delete;
5831 }; // LeafData<ValueMask>
5832 
5833 // --------------------------> LeafIndexBase <------------------------------------
5834 
5835 // Partial template specialization of LeafData with ValueIndex
5836 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5837 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafIndexBase
5838 {
5839  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5840  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5841  using ValueType = uint64_t;
5842  using FloatType = uint64_t;
5843  using ArrayType = void; // type used for the internal mValue array - void means missing
5844  static constexpr bool FIXED_SIZE = true;
5845 
5846  CoordT mBBoxMin; // 12B.
5847  uint8_t mBBoxDif[3]; // 3B.
5848  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5849  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5850  uint64_t mOffset, mPrefixSum; // 8B offset to first value in this leaf node and 9-bit prefix sum
5851  __hostdev__ static constexpr uint32_t padding()
5852  {
5853  return sizeof(LeafIndexBase) - (12u + 3u + 1u + sizeof(MaskT<LOG2DIM>) + 2 * 8u);
5854  }
5855  __hostdev__ static uint64_t memUsage() { return sizeof(LeafIndexBase); }
5856  __hostdev__ bool hasStats() const { return mFlags & (uint8_t(1) << 4); }
5857  // return the offset to the first value indexed by this leaf node
5858  __hostdev__ const uint64_t& firstOffset() const { return mOffset; }
5859  __hostdev__ void setMin(const ValueType&) {} // no-op
5860  __hostdev__ void setMax(const ValueType&) {} // no-op
5861  __hostdev__ void setAvg(const FloatType&) {} // no-op
5862  __hostdev__ void setDev(const FloatType&) {} // no-op
5863  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5864  template<typename T>
5865  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
5866 }; // LeafIndexBase
5867 
5868 // --------------------------> LeafData<ValueIndex> <------------------------------------
5869 
5870 // Partial template specialization of LeafData with ValueIndex
5871 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5872 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<ValueIndex, CoordT, MaskT, LOG2DIM>
5873  : public LeafIndexBase<CoordT, MaskT, LOG2DIM>
5874 {
5877  // return the total number of values indexed by this leaf node, excluding the optional 4 stats
5878  __hostdev__ static uint32_t valueCount() { return uint32_t(512); } // 8^3 = 2^9
5879  // return the offset to the last value indexed by this leaf node (disregarding optional stats)
5880  __hostdev__ uint64_t lastOffset() const { return BaseT::mOffset + 511u; } // 2^9 - 1
5881  // if stats are available, they are always placed after the last voxel value in this leaf node
5882  __hostdev__ uint64_t getMin() const { return this->hasStats() ? BaseT::mOffset + 512u : 0u; }
5883  __hostdev__ uint64_t getMax() const { return this->hasStats() ? BaseT::mOffset + 513u : 0u; }
5884  __hostdev__ uint64_t getAvg() const { return this->hasStats() ? BaseT::mOffset + 514u : 0u; }
5885  __hostdev__ uint64_t getDev() const { return this->hasStats() ? BaseT::mOffset + 515u : 0u; }
5886  __hostdev__ uint64_t getValue(uint32_t i) const { return BaseT::mOffset + i; } // dense leaf node with active and inactive voxels
5887 
5888  /// @brief This class cannot be constructed or deleted
5889  LeafData() = delete;
5890  LeafData(const LeafData&) = delete;
5891  LeafData& operator=(const LeafData&) = delete;
5892  ~LeafData() = delete;
5893 }; // LeafData<ValueIndex>
5894 
5895 // --------------------------> LeafData<ValueOnIndex> <------------------------------------
5896 
5897 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5898 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<ValueOnIndex, CoordT, MaskT, LOG2DIM>
5899  : public LeafIndexBase<CoordT, MaskT, LOG2DIM>
5900 {
5903  __hostdev__ uint32_t valueCount() const
5904  {
5905  return CountOn(BaseT::mValueMask.words()[7]) + (BaseT::mPrefixSum >> 54u & 511u); // last 9 bits of mPrefixSum do not account for the last word in mValueMask
5906  }
5907  __hostdev__ uint64_t lastOffset() const { return BaseT::mOffset + this->valueCount() - 1u; }
5908  __hostdev__ uint64_t getMin() const { return this->hasStats() ? this->lastOffset() + 1u : 0u; }
5909  __hostdev__ uint64_t getMax() const { return this->hasStats() ? this->lastOffset() + 2u : 0u; }
5910  __hostdev__ uint64_t getAvg() const { return this->hasStats() ? this->lastOffset() + 3u : 0u; }
5911  __hostdev__ uint64_t getDev() const { return this->hasStats() ? this->lastOffset() + 4u : 0u; }
5912  __hostdev__ uint64_t getValue(uint32_t i) const
5913  {
5914  //return mValueMask.isOn(i) ? mOffset + mValueMask.countOn(i) : 0u;// for debugging
5915  uint32_t n = i >> 6;
5916  const uint64_t w = BaseT::mValueMask.words()[n], mask = uint64_t(1) << (i & 63u);
5917  if (!(w & mask)) return uint64_t(0); // if i'th value is inactive return offset to background value
5918  uint64_t sum = BaseT::mOffset + CountOn(w & (mask - 1u));
5919  if (n--) sum += BaseT::mPrefixSum >> (9u * n) & 511u;
5920  return sum;
5921  }
5922 
5923  /// @brief This class cannot be constructed or deleted
5924  LeafData() = delete;
5925  LeafData(const LeafData&) = delete;
5926  LeafData& operator=(const LeafData&) = delete;
5927  ~LeafData() = delete;
5928 }; // LeafData<ValueOnIndex>
5929 
5930 // --------------------------> LeafData<ValueIndexMask> <------------------------------------
5931 
5932 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5933 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<ValueIndexMask, CoordT, MaskT, LOG2DIM>
5934  : public LeafData<ValueIndex, CoordT, MaskT, LOG2DIM>
5935 {
5937  MaskT<LOG2DIM> mMask;
5938  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5939  __hostdev__ bool isMaskOn(uint32_t offset) const { return mMask.isOn(offset); }
5940  __hostdev__ void setMask(uint32_t offset, bool v) { mMask.set(offset, v); }
5941 }; // LeafData<ValueIndexMask>
5942 
5943 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5944 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<ValueOnIndexMask, CoordT, MaskT, LOG2DIM>
5945  : public LeafData<ValueOnIndex, CoordT, MaskT, LOG2DIM>
5946 {
5948  MaskT<LOG2DIM> mMask;
5949  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5950  __hostdev__ bool isMaskOn(uint32_t offset) const { return mMask.isOn(offset); }
5951  __hostdev__ void setMask(uint32_t offset, bool v) { mMask.set(offset, v); }
5952 }; // LeafData<ValueOnIndexMask>
5953 
5954 // --------------------------> LeafData<Point> <------------------------------------
5955 
5956 template<typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
5957 struct NANOVDB_ALIGN(NANOVDB_DATA_ALIGNMENT) LeafData<Point, CoordT, MaskT, LOG2DIM>
5958 {
5959  static_assert(sizeof(CoordT) == sizeof(Coord), "Mismatching sizeof");
5960  static_assert(sizeof(MaskT<LOG2DIM>) == sizeof(Mask<LOG2DIM>), "Mismatching sizeof");
5961  using ValueType = uint64_t;
5962  using BuildType = Point;
5964  using ArrayType = uint16_t; // type used for the internal mValue array
5965  static constexpr bool FIXED_SIZE = true;
5966 
5967  CoordT mBBoxMin; // 12B.
5968  uint8_t mBBoxDif[3]; // 3B.
5969  uint8_t mFlags; // 1B. bit0: skip render?, bit1: has bbox?, bit3: unused, bit4: has stats, bits5,6,7: bit-width for FpN
5970  MaskT<LOG2DIM> mValueMask; // LOG2DIM(3): 64B.
5971 
5972  uint64_t mOffset; // 8B
5973  uint64_t mPointCount; // 8B
5974  alignas(32) uint16_t mValues[1u << 3 * LOG2DIM]; // 1KB
5975  // no padding
5976 
5977  /// @brief Return padding of this class in bytes, due to aliasing and 32B alignment
5978  ///
5979  /// @note The extra bytes are not necessarily at the end, but can come from aliasing of individual data members.
5980  __hostdev__ static constexpr uint32_t padding()
5981  {
5982  return sizeof(LeafData) - (12u + 3u + 1u + sizeof(MaskT<LOG2DIM>) + 2 * 8u + (1u << 3 * LOG2DIM) * 2u);
5983  }
5984  __hostdev__ static uint64_t memUsage() { return sizeof(LeafData); }
5985 
5986  __hostdev__ uint64_t offset() const { return mOffset; }
5987  __hostdev__ uint64_t pointCount() const { return mPointCount; }
5988  __hostdev__ uint64_t first(uint32_t i) const { return i ? uint64_t(mValues[i - 1u]) + mOffset : mOffset; }
5989  __hostdev__ uint64_t last(uint32_t i) const { return uint64_t(mValues[i]) + mOffset; }
5990  __hostdev__ uint64_t getValue(uint32_t i) const { return uint64_t(mValues[i]); }
5991  __hostdev__ void setValueOnly(uint32_t offset, uint16_t value) { mValues[offset] = value; }
5992  __hostdev__ void setValue(uint32_t offset, uint16_t value)
5993  {
5994  mValueMask.setOn(offset);
5995  mValues[offset] = value;
5996  }
5997  __hostdev__ void setOn(uint32_t offset) { mValueMask.setOn(offset); }
5998 
5999  __hostdev__ ValueType getMin() const { return mOffset; }
6000  __hostdev__ ValueType getMax() const { return mPointCount; }
6001  __hostdev__ FloatType getAvg() const { return 0.0f; }
6002  __hostdev__ FloatType getDev() const { return 0.0f; }
6003 
6004  __hostdev__ void setMin(const ValueType&) {}
6005  __hostdev__ void setMax(const ValueType&) {}
6006  __hostdev__ void setAvg(const FloatType&) {}
6007  __hostdev__ void setDev(const FloatType&) {}
6008 
6009  template<typename T>
6010  __hostdev__ void setOrigin(const T& ijk) { mBBoxMin = ijk; }
6011 
6012  /// @brief This class cannot be constructed or deleted
6013  LeafData() = delete;
6014  LeafData(const LeafData&) = delete;
6015  LeafData& operator=(const LeafData&) = delete;
6016  ~LeafData() = delete;
6017 }; // LeafData<Point>
6018 
6019 // --------------------------> LeafNode<T> <------------------------------------
6020 
6021 /// @brief Leaf nodes of the VDB tree. (defaults to 8x8x8 = 512 voxels)
6022 template<typename BuildT,
6023  typename CoordT = Coord,
6024  template<uint32_t> class MaskT = Mask,
6025  uint32_t Log2Dim = 3>
6026 class LeafNode : public LeafData<BuildT, CoordT, MaskT, Log2Dim>
6027 {
6028 public:
6030  {
6031  static constexpr uint32_t TOTAL = 0;
6032  static constexpr uint32_t DIM = 1;
6033  __hostdev__ static uint32_t dim() { return 1u; }
6034  }; // Voxel
6037  using ValueType = typename DataType::ValueType;
6038  using FloatType = typename DataType::FloatType;
6039  using BuildType = typename DataType::BuildType;
6040  using CoordType = CoordT;
6041  static constexpr bool FIXED_SIZE = DataType::FIXED_SIZE;
6042  template<uint32_t LOG2>
6043  using MaskType = MaskT<LOG2>;
6044  template<bool ON>
6045  using MaskIterT = typename Mask<Log2Dim>::template Iterator<ON>;
6046 
6047  /// @brief Visits all active values in a leaf node
6048  class ValueOnIterator : public MaskIterT<true>
6049  {
6050  using BaseT = MaskIterT<true>;
6051  const LeafNode* mParent;
6052 
6053  public:
6055  : BaseT()
6056  , mParent(nullptr)
6057  {
6058  }
6060  : BaseT(parent->data()->mValueMask.beginOn())
6061  , mParent(parent)
6062  {
6063  }
6064  ValueOnIterator& operator=(const ValueOnIterator&) = default;
6066  {
6067  NANOVDB_ASSERT(*this);
6068  return mParent->getValue(BaseT::pos());
6069  }
6070  __hostdev__ CoordT getCoord() const
6071  {
6072  NANOVDB_ASSERT(*this);
6073  return mParent->offsetToGlobalCoord(BaseT::pos());
6074  }
6075  }; // Member class ValueOnIterator
6076 
6077  __hostdev__ ValueOnIterator beginValueOn() const { return ValueOnIterator(this); }
6078  __hostdev__ ValueOnIterator cbeginValueOn() const { return ValueOnIterator(this); }
6079 
6080  /// @brief Visits all inactive values in a leaf node
6081  class ValueOffIterator : public MaskIterT<false>
6082  {
6083  using BaseT = MaskIterT<false>;
6084  const LeafNode* mParent;
6085 
6086  public:
6088  : BaseT()
6089  , mParent(nullptr)
6090  {
6091  }
6093  : BaseT(parent->data()->mValueMask.beginOff())
6094  , mParent(parent)
6095  {
6096  }
6097  ValueOffIterator& operator=(const ValueOffIterator&) = default;
6099  {
6100  NANOVDB_ASSERT(*this);
6101  return mParent->getValue(BaseT::pos());
6102  }
6103  __hostdev__ CoordT getCoord() const
6104  {
6105  NANOVDB_ASSERT(*this);
6106  return mParent->offsetToGlobalCoord(BaseT::pos());
6107  }
6108  }; // Member class ValueOffIterator
6109 
6110  __hostdev__ ValueOffIterator beginValueOff() const { return ValueOffIterator(this); }
6111  __hostdev__ ValueOffIterator cbeginValueOff() const { return ValueOffIterator(this); }
6112 
6113  /// @brief Visits all values in a leaf node, i.e. both active and inactive values
6115  {
6116  const LeafNode* mParent;
6117  uint32_t mPos;
6118 
6119  public:
6121  : mParent(nullptr)
6122  , mPos(1u << 3 * Log2Dim)
6123  {
6124  }
6126  : mParent(parent)
6127  , mPos(0)
6128  {
6129  NANOVDB_ASSERT(parent);
6130  }
6131  ValueIterator& operator=(const ValueIterator&) = default;
6133  {
6134  NANOVDB_ASSERT(*this);
6135  return mParent->getValue(mPos);
6136  }
6137  __hostdev__ CoordT getCoord() const
6138  {
6139  NANOVDB_ASSERT(*this);
6140  return mParent->offsetToGlobalCoord(mPos);
6141  }
6142  __hostdev__ bool isActive() const
6143  {
6144  NANOVDB_ASSERT(*this);
6145  return mParent->isActive(mPos);
6146  }
6147  __hostdev__ operator bool() const { return mPos < (1u << 3 * Log2Dim); }
6149  {
6150  ++mPos;
6151  return *this;
6152  }
6154  {
6155  auto tmp = *this;
6156  ++(*this);
6157  return tmp;
6158  }
6159  }; // Member class ValueIterator
6160 
6161  __hostdev__ ValueIterator beginValue() const { return ValueIterator(this); }
6162  __hostdev__ ValueIterator cbeginValueAll() const { return ValueIterator(this); }
6163 
6164  static_assert(is_same<ValueType, typename BuildToValueMap<BuildType>::Type>::value, "Mismatching BuildType");
6165  static constexpr uint32_t LOG2DIM = Log2Dim;
6166  static constexpr uint32_t TOTAL = LOG2DIM; // needed by parent nodes
6167  static constexpr uint32_t DIM = 1u << TOTAL; // number of voxels along each axis of this node
6168  static constexpr uint32_t SIZE = 1u << 3 * LOG2DIM; // total number of voxels represented by this node
6169  static constexpr uint32_t MASK = (1u << LOG2DIM) - 1u; // mask for bit operations
6170  static constexpr uint32_t LEVEL = 0; // level 0 = leaf
6171  static constexpr uint64_t NUM_VALUES = uint64_t(1) << (3 * TOTAL); // total voxel count represented by this node
6172 
6173  __hostdev__ DataType* data() { return reinterpret_cast<DataType*>(this); }
6174 
6175  __hostdev__ const DataType* data() const { return reinterpret_cast<const DataType*>(this); }
6176 
6177  /// @brief Return a const reference to the bit mask of active voxels in this leaf node
6178  __hostdev__ const MaskType<LOG2DIM>& valueMask() const { return DataType::mValueMask; }
6179  __hostdev__ const MaskType<LOG2DIM>& getValueMask() const { return DataType::mValueMask; }
6180 
6181  /// @brief Return a const reference to the minimum active value encoded in this leaf node
6182  __hostdev__ ValueType minimum() const { return DataType::getMin(); }
6183 
6184  /// @brief Return a const reference to the maximum active value encoded in this leaf node
6185  __hostdev__ ValueType maximum() const { return DataType::getMax(); }
6186 
6187  /// @brief Return a const reference to the average of all the active values encoded in this leaf node
6188  __hostdev__ FloatType average() const { return DataType::getAvg(); }
6189 
6190  /// @brief Return the variance of all the active values encoded in this leaf node
6191  __hostdev__ FloatType variance() const { return Pow2(DataType::getDev()); }
6192 
6193  /// @brief Return a const reference to the standard deviation of all the active values encoded in this leaf node
6194  __hostdev__ FloatType stdDeviation() const { return DataType::getDev(); }
6195 
6196  __hostdev__ uint8_t flags() const { return DataType::mFlags; }
6197 
6198  /// @brief Return the origin in index space of this leaf node
6199  __hostdev__ CoordT origin() const { return DataType::mBBoxMin & ~MASK; }
6200 
6201  /// @brief Compute the local coordinates from a linear offset
6202  /// @param n Linear offset into this nodes dense table
6203  /// @return Local (vs global) 3D coordinates
6204  __hostdev__ static CoordT OffsetToLocalCoord(uint32_t n)
6205  {
6206  NANOVDB_ASSERT(n < SIZE);
6207  const uint32_t m = n & ((1 << 2 * LOG2DIM) - 1);
6208  return CoordT(n >> 2 * LOG2DIM, m >> LOG2DIM, m & MASK);
6209  }
6210 
6211  /// @brief Converts (in place) a local index coordinate to a global index coordinate
6212  __hostdev__ void localToGlobalCoord(Coord& ijk) const { ijk += this->origin(); }
6213 
6214  __hostdev__ CoordT offsetToGlobalCoord(uint32_t n) const
6215  {
6216  return OffsetToLocalCoord(n) + this->origin();
6217  }
6218 
6219  /// @brief Return the dimension, in index space, of this leaf node (typically 8 as for openvdb leaf nodes!)
6220  __hostdev__ static uint32_t dim() { return 1u << LOG2DIM; }
6221 
6222  /// @brief Return the bounding box in index space of active values in this leaf node
6224  {
6225  BBox<CoordT> bbox(DataType::mBBoxMin, DataType::mBBoxMin);
6226  if (this->hasBBox()) {
6227  bbox.max()[0] += DataType::mBBoxDif[0];
6228  bbox.max()[1] += DataType::mBBoxDif[1];
6229  bbox.max()[2] += DataType::mBBoxDif[2];
6230  } else { // very rare case
6231  bbox = BBox<CoordT>(); // invalid
6232  }
6233  return bbox;
6234  }
6235 
6236  /// @brief Return the total number of voxels (e.g. values) encoded in this leaf node
6237  __hostdev__ static uint32_t voxelCount() { return 1u << (3 * LOG2DIM); }
6238 
6239  __hostdev__ static uint32_t padding() { return DataType::padding(); }
6240 
6241  /// @brief return memory usage in bytes for the leaf node
6242  __hostdev__ uint64_t memUsage() const { return DataType::memUsage(); }
6243 
6244  /// @brief This class cannot be constructed or deleted
6245  LeafNode() = delete;
6246  LeafNode(const LeafNode&) = delete;
6247  LeafNode& operator=(const LeafNode&) = delete;
6248  ~LeafNode() = delete;
6249 
6250  /// @brief Return the voxel value at the given offset.
6251  __hostdev__ ValueType getValue(uint32_t offset) const { return DataType::getValue(offset); }
6252 
6253  /// @brief Return the voxel value at the given coordinate.
6254  __hostdev__ ValueType getValue(const CoordT& ijk) const { return DataType::getValue(CoordToOffset(ijk)); }
6255 
6256  /// @brief Return the first value in this leaf node.
6257  __hostdev__ ValueType getFirstValue() const { return this->getValue(0); }
6258  /// @brief Return the last value in this leaf node.
6259  __hostdev__ ValueType getLastValue() const { return this->getValue(SIZE - 1); }
6260 
6261  /// @brief Sets the value at the specified location and activate its state.
6262  ///
6263  /// @note This is safe since it does not change the topology of the tree (unlike setValue methods on the other nodes)
6264  __hostdev__ void setValue(const CoordT& ijk, const ValueType& v) { DataType::setValue(CoordToOffset(ijk), v); }
6265 
6266  /// @brief Sets the value at the specified location but leaves its state unchanged.
6267  ///
6268  /// @note This is safe since it does not change the topology of the tree (unlike setValue methods on the other nodes)
6269  __hostdev__ void setValueOnly(uint32_t offset, const ValueType& v) { DataType::setValueOnly(offset, v); }
6270  __hostdev__ void setValueOnly(const CoordT& ijk, const ValueType& v) { DataType::setValueOnly(CoordToOffset(ijk), v); }
6271 
6272  /// @brief Return @c true if the voxel value at the given coordinate is active.
6273  __hostdev__ bool isActive(const CoordT& ijk) const { return DataType::mValueMask.isOn(CoordToOffset(ijk)); }
6274  __hostdev__ bool isActive(uint32_t n) const { return DataType::mValueMask.isOn(n); }
6275 
6276  /// @brief Return @c true if any of the voxel value are active in this leaf node.
6277  __hostdev__ bool isActive() const
6278  {
6279  //NANOVDB_ASSERT( bool(DataType::mFlags & uint8_t(2)) != DataType::mValueMask.isOff() );
6280  //return DataType::mFlags & uint8_t(2);
6281  return !DataType::mValueMask.isOff();
6282  }
6283 
6284  __hostdev__ bool hasBBox() const { return DataType::mFlags & uint8_t(2); }
6285 
6286  /// @brief Return @c true if the voxel value at the given coordinate is active and updates @c v with the value.
6287  __hostdev__ bool probeValue(const CoordT& ijk, ValueType& v) const
6288  {
6289  const uint32_t n = CoordToOffset(ijk);
6290  v = DataType::getValue(n);
6291  return DataType::mValueMask.isOn(n);
6292  }
6293 
6294  __hostdev__ const LeafNode* probeLeaf(const CoordT&) const { return this; }
6295 
6296  /// @brief Return the linear offset corresponding to the given coordinate
6297  __hostdev__ static uint32_t CoordToOffset(const CoordT& ijk)
6298  {
6299  return ((ijk[0] & MASK) << (2 * LOG2DIM)) | ((ijk[1] & MASK) << LOG2DIM) | (ijk[2] & MASK);
6300  }
6301 
6302  /// @brief Updates the local bounding box of active voxels in this node. Return true if bbox was updated.
6303  ///
6304  /// @warning It assumes that the origin and value mask have already been set.
6305  ///
6306  /// @details This method is based on few (intrinsic) bit operations and hence is relatively fast.
6307  /// However, it should only only be called if either the value mask has changed or if the
6308  /// active bounding box is still undefined. e.g. during construction of this node.
6309  __hostdev__ bool updateBBox();
6310 
6311  template<typename OpT, typename... ArgsT>
6312  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
6313  {
6314  return OpT::get(*this, CoordToOffset(ijk), args...);
6315  }
6316 
6317  template<typename OpT, typename... ArgsT>
6318  __hostdev__ auto get(const uint32_t n, ArgsT&&... args) const
6319  {
6320  return OpT::get(*this, n, args...);
6321  }
6322 
6323  template<typename OpT, typename... ArgsT>
6324  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args)
6325  {
6326  return OpT::set(*this, CoordToOffset(ijk), args...);
6327  }
6328 
6329  template<typename OpT, typename... ArgsT>
6330  __hostdev__ auto set(const uint32_t n, ArgsT&&... args)
6331  {
6332  return OpT::set(*this, n, args...);
6333  }
6334 
6335 private:
6336  static_assert(sizeof(DataType) % NANOVDB_DATA_ALIGNMENT == 0, "sizeof(LeafData) is misaligned");
6337 
6338  template<typename, int, int, int>
6339  friend class ReadAccessor;
6340 
6341  template<typename>
6342  friend class RootNode;
6343  template<typename, uint32_t>
6344  friend class InternalNode;
6345 
6346 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
6347  /// @brief Private method to return a voxel value and update a (dummy) ReadAccessor
6348  template<typename AccT>
6349  __hostdev__ ValueType getValueAndCache(const CoordT& ijk, const AccT&) const { return this->getValue(ijk); }
6350 
6351  /// @brief Return the node information.
6352  template<typename AccT>
6353  __hostdev__ typename AccT::NodeInfo getNodeInfoAndCache(const CoordType& /*ijk*/, const AccT& /*acc*/) const
6354  {
6355  using NodeInfoT = typename AccT::NodeInfo;
6356  return NodeInfoT{LEVEL, this->dim(), this->minimum(), this->maximum(), this->average(), this->stdDeviation(), this->bbox()[0], this->bbox()[1]};
6357  }
6358 
6359  template<typename AccT>
6360  __hostdev__ bool isActiveAndCache(const CoordT& ijk, const AccT&) const { return this->isActive(ijk); }
6361 
6362  template<typename AccT>
6363  __hostdev__ bool probeValueAndCache(const CoordT& ijk, ValueType& v, const AccT&) const { return this->probeValue(ijk, v); }
6364 
6365  template<typename AccT>
6366  __hostdev__ const LeafNode* probeLeafAndCache(const CoordT&, const AccT&) const { return this; }
6367 #endif
6368 
6369  template<typename RayT, typename AccT>
6370  __hostdev__ uint32_t getDimAndCache(const CoordT&, const RayT& /*ray*/, const AccT&) const
6371  {
6372  if (DataType::mFlags & uint8_t(1u))
6373  return this->dim(); // skip this node if the 1st bit is set
6374 
6375  //if (!ray.intersects( this->bbox() )) return 1 << LOG2DIM;
6376  return ChildNodeType::dim();
6377  }
6378 
6379  template<typename OpT, typename AccT, typename... ArgsT>
6380  __hostdev__ auto
6381  //__hostdev__ decltype(OpT::get(std::declval<const LeafNode&>(), std::declval<uint32_t>(), std::declval<ArgsT>()...))
6382  getAndCache(const CoordType& ijk, const AccT&, ArgsT&&... args) const
6383  {
6384  return OpT::get(*this, CoordToOffset(ijk), args...);
6385  }
6386 
6387  template<typename OpT, typename AccT, typename... ArgsT>
6388  //__hostdev__ auto // occasionally fails with NVCC
6389  __hostdev__ decltype(OpT::set(std::declval<LeafNode&>(), std::declval<uint32_t>(), std::declval<ArgsT>()...))
6390  setAndCache(const CoordType& ijk, const AccT&, ArgsT&&... args)
6391  {
6392  return OpT::set(*this, CoordToOffset(ijk), args...);
6393  }
6394 
6395 }; // LeafNode class
6396 
6397 // --------------------------> LeafNode<T>::updateBBox <------------------------------------
6398 
6399 template<typename ValueT, typename CoordT, template<uint32_t> class MaskT, uint32_t LOG2DIM>
6401 {
6402  static_assert(LOG2DIM == 3, "LeafNode::updateBBox: only supports LOGDIM = 3!");
6403  if (DataType::mValueMask.isOff()) {
6404  DataType::mFlags &= ~uint8_t(2); // set 2nd bit off, which indicates that this nodes has no bbox
6405  return false;
6406  }
6407  auto update = [&](uint32_t min, uint32_t max, int axis) {
6408  NANOVDB_ASSERT(min <= max && max < 8);
6409  DataType::mBBoxMin[axis] = (DataType::mBBoxMin[axis] & ~MASK) + int(min);
6410  DataType::mBBoxDif[axis] = uint8_t(max - min);
6411  };
6412  uint64_t *w = DataType::mValueMask.words(), word64 = *w;
6413  uint32_t Xmin = word64 ? 0u : 8u, Xmax = Xmin;
6414  for (int i = 1; i < 8; ++i) { // last loop over 8 64 bit words
6415  if (w[i]) { // skip if word has no set bits
6416  word64 |= w[i]; // union 8 x 64 bits words into one 64 bit word
6417  if (Xmin == 8)
6418  Xmin = i; // only set once
6419  Xmax = i;
6420  }
6421  }
6422  NANOVDB_ASSERT(word64);
6423  update(Xmin, Xmax, 0);
6424  update(FindLowestOn(word64) >> 3, FindHighestOn(word64) >> 3, 1);
6425  const uint32_t *p = reinterpret_cast<const uint32_t*>(&word64), word32 = p[0] | p[1];
6426  const uint16_t *q = reinterpret_cast<const uint16_t*>(&word32), word16 = q[0] | q[1];
6427  const uint8_t * b = reinterpret_cast<const uint8_t*>(&word16), byte = b[0] | b[1];
6428  NANOVDB_ASSERT(byte);
6429  update(FindLowestOn(static_cast<uint32_t>(byte)), FindHighestOn(static_cast<uint32_t>(byte)), 2);
6430  DataType::mFlags |= uint8_t(2); // set 2nd bit on, which indicates that this nodes has a bbox
6431  return true;
6432 } // LeafNode::updateBBox
6433 
6434 // --------------------------> Template specializations and traits <------------------------------------
6435 
6436 /// @brief Template specializations to the default configuration used in OpenVDB:
6437 /// Root -> 32^3 -> 16^3 -> 8^3
6438 template<typename BuildT>
6440 template<typename BuildT>
6442 template<typename BuildT>
6444 template<typename BuildT>
6446 template<typename BuildT>
6448 template<typename BuildT>
6450 
6451 /// @brief Trait to map from LEVEL to node type
6452 template<typename BuildT, int LEVEL>
6453 struct NanoNode;
6454 
6455 // Partial template specialization of above Node struct
6456 template<typename BuildT>
6457 struct NanoNode<BuildT, 0>
6458 {
6461 };
6462 template<typename BuildT>
6463 struct NanoNode<BuildT, 1>
6464 {
6467 };
6468 template<typename BuildT>
6469 struct NanoNode<BuildT, 2>
6470 {
6473 };
6474 template<typename BuildT>
6475 struct NanoNode<BuildT, 3>
6476 {
6479 };
6480 
6501 
6523 
6524 // --------------------------> ReadAccessor <------------------------------------
6525 
6526 /// @brief A read-only value accessor with three levels of node caching. This allows for
6527 /// inverse tree traversal during lookup, which is on average significantly faster
6528 /// than calling the equivalent method on the tree (i.e. top-down traversal).
6529 ///
6530 /// @note By virtue of the fact that a value accessor accelerates random access operations
6531 /// by re-using cached access patterns, this access should be reused for multiple access
6532 /// operations. In other words, never create an instance of this accessor for a single
6533 /// access only. In general avoid single access operations with this accessor, and
6534 /// if that is not possible call the corresponding method on the tree instead.
6535 ///
6536 /// @warning Since this ReadAccessor internally caches raw pointers to the nodes of the tree
6537 /// structure, it is not safe to copy between host and device, or even to share among
6538 /// multiple threads on the same host or device. However, it is light-weight so simple
6539 /// instantiate one per thread (on the host and/or device).
6540 ///
6541 /// @details Used to accelerated random access into a VDB tree. Provides on average
6542 /// O(1) random access operations by means of inverse tree traversal,
6543 /// which amortizes the non-const time complexity of the root node.
6544 
6545 template<typename BuildT>
6546 class ReadAccessor<BuildT, -1, -1, -1>
6547 {
6548  using GridT = NanoGrid<BuildT>; // grid
6549  using TreeT = NanoTree<BuildT>; // tree
6550  using RootT = NanoRoot<BuildT>; // root node
6551  using LeafT = NanoLeaf<BuildT>; // Leaf node
6552  using FloatType = typename RootT::FloatType;
6553  using CoordValueType = typename RootT::CoordType::ValueType;
6554 
6555  mutable const RootT* mRoot; // 8 bytes (mutable to allow for access methods to be const)
6556 public:
6557  using BuildType = BuildT;
6558  using ValueType = typename RootT::ValueType;
6559  using CoordType = typename RootT::CoordType;
6560 
6561  static const int CacheLevels = 0;
6562 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
6563  struct NodeInfo
6564  {
6565  uint32_t mLevel; // 4B
6566  uint32_t mDim; // 4B
6567  ValueType mMinimum; // typically 4B
6568  ValueType mMaximum; // typically 4B
6569  FloatType mAverage; // typically 4B
6570  FloatType mStdDevi; // typically 4B
6571  CoordType mBBoxMin; // 3*4B
6572  CoordType mBBoxMax; // 3*4B
6573  };
6574 #endif
6575  /// @brief Constructor from a root node
6577  : mRoot{&root}
6578  {
6579  }
6580 
6581  /// @brief Constructor from a grid
6582  __hostdev__ ReadAccessor(const GridT& grid)
6583  : ReadAccessor(grid.tree().root())
6584  {
6585  }
6586 
6587  /// @brief Constructor from a tree
6589  : ReadAccessor(tree.root())
6590  {
6591  }
6592 
6593  /// @brief Reset this access to its initial state, i.e. with an empty cache
6594  /// @node Noop since this template specialization has no cache
6595  __hostdev__ void clear() {}
6596 
6597  __hostdev__ const RootT& root() const { return *mRoot; }
6598 
6599  /// @brief Defaults constructors
6600  ReadAccessor(const ReadAccessor&) = default;
6601  ~ReadAccessor() = default;
6602  ReadAccessor& operator=(const ReadAccessor&) = default;
6603 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
6605  {
6606  return this->template get<GetValue<BuildT>>(ijk);
6607  }
6608  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6609  __hostdev__ ValueType operator()(const CoordType& ijk) const { return this->template get<GetValue<BuildT>>(ijk); }
6610  __hostdev__ ValueType operator()(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6611  __hostdev__ auto getNodeInfo(const CoordType& ijk) const { return this->template get<GetNodeInfo<BuildT>>(ijk); }
6612  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildT>>(ijk); }
6613  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildT>>(ijk, v); }
6614  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildT>>(ijk); }
6615 #else // NANOVDB_NEW_ACCESSOR_METHODS
6616  __hostdev__ ValueType getValue(const CoordType& ijk) const
6617  {
6618  return mRoot->getValueAndCache(ijk, *this);
6619  }
6620  __hostdev__ ValueType getValue(int i, int j, int k) const
6621  {
6622  return this->getValue(CoordType(i, j, k));
6623  }
6624  __hostdev__ ValueType operator()(const CoordType& ijk) const
6625  {
6626  return this->getValue(ijk);
6627  }
6628  __hostdev__ ValueType operator()(int i, int j, int k) const
6629  {
6630  return this->getValue(CoordType(i, j, k));
6631  }
6632 
6633  __hostdev__ NodeInfo getNodeInfo(const CoordType& ijk) const
6634  {
6635  return mRoot->getNodeInfoAndCache(ijk, *this);
6636  }
6637 
6638  __hostdev__ bool isActive(const CoordType& ijk) const
6639  {
6640  return mRoot->isActiveAndCache(ijk, *this);
6641  }
6642 
6643  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
6644  {
6645  return mRoot->probeValueAndCache(ijk, v, *this);
6646  }
6647 
6648  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const
6649  {
6650  return mRoot->probeLeafAndCache(ijk, *this);
6651  }
6652 #endif // NANOVDB_NEW_ACCESSOR_METHODS
6653  template<typename RayT>
6654  __hostdev__ uint32_t getDim(const CoordType& ijk, const RayT& ray) const
6655  {
6656  return mRoot->getDimAndCache(ijk, ray, *this);
6657  }
6658  template<typename OpT, typename... ArgsT>
6659  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
6660  {
6661  return mRoot->template get<OpT>(ijk, args...);
6662  }
6663 
6664  template<typename OpT, typename... ArgsT>
6665  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args) const
6666  {
6667  return const_cast<RootT*>(mRoot)->template set<OpT>(ijk, args...);
6668  }
6669 
6670 private:
6671  /// @brief Allow nodes to insert themselves into the cache.
6672  template<typename>
6673  friend class RootNode;
6674  template<typename, uint32_t>
6675  friend class InternalNode;
6676  template<typename, typename, template<uint32_t> class, uint32_t>
6677  friend class LeafNode;
6678 
6679  /// @brief No-op
6680  template<typename NodeT>
6681  __hostdev__ void insert(const CoordType&, const NodeT*) const {}
6682 }; // ReadAccessor<ValueT, -1, -1, -1> class
6683 
6684 /// @brief Node caching at a single tree level
6685 template<typename BuildT, int LEVEL0>
6686 class ReadAccessor<BuildT, LEVEL0, -1, -1> //e.g. 0, 1, 2
6687 {
6688  static_assert(LEVEL0 >= 0 && LEVEL0 <= 2, "LEVEL0 should be 0, 1, or 2");
6689 
6690  using GridT = NanoGrid<BuildT>; // grid
6691  using TreeT = NanoTree<BuildT>;
6692  using RootT = NanoRoot<BuildT>; // root node
6693  using LeafT = NanoLeaf<BuildT>; // Leaf node
6694  using NodeT = typename NodeTrait<TreeT, LEVEL0>::type;
6695  using CoordT = typename RootT::CoordType;
6696  using ValueT = typename RootT::ValueType;
6697 
6698  using FloatType = typename RootT::FloatType;
6699  using CoordValueType = typename RootT::CoordT::ValueType;
6700 
6701  // All member data are mutable to allow for access methods to be const
6702  mutable CoordT mKey; // 3*4 = 12 bytes
6703  mutable const RootT* mRoot; // 8 bytes
6704  mutable const NodeT* mNode; // 8 bytes
6705 
6706 public:
6707  using BuildType = BuildT;
6708  using ValueType = ValueT;
6709  using CoordType = CoordT;
6710 
6711  static const int CacheLevels = 1;
6712 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
6713  using NodeInfo = typename ReadAccessor<ValueT, -1, -1, -1>::NodeInfo;
6714 #endif
6715  /// @brief Constructor from a root node
6717  : mKey(CoordType::max())
6718  , mRoot(&root)
6719  , mNode(nullptr)
6720  {
6721  }
6722 
6723  /// @brief Constructor from a grid
6725  : ReadAccessor(grid.tree().root())
6726  {
6727  }
6728 
6729  /// @brief Constructor from a tree
6731  : ReadAccessor(tree.root())
6732  {
6733  }
6734 
6735  /// @brief Reset this access to its initial state, i.e. with an empty cache
6737  {
6738  mKey = CoordType::max();
6739  mNode = nullptr;
6740  }
6741 
6742  __hostdev__ const RootT& root() const { return *mRoot; }
6743 
6744  /// @brief Defaults constructors
6745  ReadAccessor(const ReadAccessor&) = default;
6746  ~ReadAccessor() = default;
6747  ReadAccessor& operator=(const ReadAccessor&) = default;
6748 
6749  __hostdev__ bool isCached(const CoordType& ijk) const
6750  {
6751  return (ijk[0] & int32_t(~NodeT::MASK)) == mKey[0] &&
6752  (ijk[1] & int32_t(~NodeT::MASK)) == mKey[1] &&
6753  (ijk[2] & int32_t(~NodeT::MASK)) == mKey[2];
6754  }
6755 
6756 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
6758  {
6759  return this->template get<GetValue<BuildT>>(ijk);
6760  }
6761  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6762  __hostdev__ ValueType operator()(const CoordType& ijk) const { return this->template get<GetValue<BuildT>>(ijk); }
6763  __hostdev__ ValueType operator()(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6764  __hostdev__ auto getNodeInfo(const CoordType& ijk) const { return this->template get<GetNodeInfo<BuildT>>(ijk); }
6765  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildT>>(ijk); }
6766  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildT>>(ijk, v); }
6767  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildT>>(ijk); }
6768 #else // NANOVDB_NEW_ACCESSOR_METHODS
6769  __hostdev__ ValueType getValue(const CoordType& ijk) const
6770  {
6771  if (this->isCached(ijk))
6772  return mNode->getValueAndCache(ijk, *this);
6773  return mRoot->getValueAndCache(ijk, *this);
6774  }
6775  __hostdev__ ValueType getValue(int i, int j, int k) const
6776  {
6777  return this->getValue(CoordType(i, j, k));
6778  }
6779  __hostdev__ ValueType operator()(const CoordType& ijk) const
6780  {
6781  return this->getValue(ijk);
6782  }
6783  __hostdev__ ValueType operator()(int i, int j, int k) const
6784  {
6785  return this->getValue(CoordType(i, j, k));
6786  }
6787 
6788  __hostdev__ NodeInfo getNodeInfo(const CoordType& ijk) const
6789  {
6790  if (this->isCached(ijk))
6791  return mNode->getNodeInfoAndCache(ijk, *this);
6792  return mRoot->getNodeInfoAndCache(ijk, *this);
6793  }
6794 
6795  __hostdev__ bool isActive(const CoordType& ijk) const
6796  {
6797  if (this->isCached(ijk))
6798  return mNode->isActiveAndCache(ijk, *this);
6799  return mRoot->isActiveAndCache(ijk, *this);
6800  }
6801 
6802  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
6803  {
6804  if (this->isCached(ijk))
6805  return mNode->probeValueAndCache(ijk, v, *this);
6806  return mRoot->probeValueAndCache(ijk, v, *this);
6807  }
6808 
6809  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const
6810  {
6811  if (this->isCached(ijk))
6812  return mNode->probeLeafAndCache(ijk, *this);
6813  return mRoot->probeLeafAndCache(ijk, *this);
6814  }
6815 #endif // NANOVDB_NEW_ACCESSOR_METHODS
6816  template<typename RayT>
6817  __hostdev__ uint32_t getDim(const CoordType& ijk, const RayT& ray) const
6818  {
6819  if (this->isCached(ijk))
6820  return mNode->getDimAndCache(ijk, ray, *this);
6821  return mRoot->getDimAndCache(ijk, ray, *this);
6822  }
6823 
6824  template<typename OpT, typename... ArgsT>
6825  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
6826  {
6827  if (this->isCached(ijk))
6828  return mNode->template getAndCache<OpT>(ijk, *this, args...);
6829  return mRoot->template getAndCache<OpT>(ijk, *this, args...);
6830  }
6831 
6832  template<typename OpT, typename... ArgsT>
6833  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args) const
6834  {
6835  if (this->isCached(ijk))
6836  return const_cast<NodeT*>(mNode)->template setAndCache<OpT>(ijk, *this, args...);
6837  return const_cast<RootT*>(mRoot)->template setAndCache<OpT>(ijk, *this, args...);
6838  }
6839 
6840 private:
6841  /// @brief Allow nodes to insert themselves into the cache.
6842  template<typename>
6843  friend class RootNode;
6844  template<typename, uint32_t>
6845  friend class InternalNode;
6846  template<typename, typename, template<uint32_t> class, uint32_t>
6847  friend class LeafNode;
6848 
6849  /// @brief Inserts a leaf node and key pair into this ReadAccessor
6850  __hostdev__ void insert(const CoordType& ijk, const NodeT* node) const
6851  {
6852  mKey = ijk & ~NodeT::MASK;
6853  mNode = node;
6854  }
6855 
6856  // no-op
6857  template<typename OtherNodeT>
6858  __hostdev__ void insert(const CoordType&, const OtherNodeT*) const {}
6859 
6860 }; // ReadAccessor<ValueT, LEVEL0>
6861 
6862 template<typename BuildT, int LEVEL0, int LEVEL1>
6863 class ReadAccessor<BuildT, LEVEL0, LEVEL1, -1> //e.g. (0,1), (1,2), (0,2)
6864 {
6865  static_assert(LEVEL0 >= 0 && LEVEL0 <= 2, "LEVEL0 must be 0, 1, 2");
6866  static_assert(LEVEL1 >= 0 && LEVEL1 <= 2, "LEVEL1 must be 0, 1, 2");
6867  static_assert(LEVEL0 < LEVEL1, "Level 0 must be lower than level 1");
6868  using GridT = NanoGrid<BuildT>; // grid
6869  using TreeT = NanoTree<BuildT>;
6870  using RootT = NanoRoot<BuildT>;
6871  using LeafT = NanoLeaf<BuildT>;
6872  using Node1T = typename NodeTrait<TreeT, LEVEL0>::type;
6873  using Node2T = typename NodeTrait<TreeT, LEVEL1>::type;
6874  using CoordT = typename RootT::CoordType;
6875  using ValueT = typename RootT::ValueType;
6876  using FloatType = typename RootT::FloatType;
6877  using CoordValueType = typename RootT::CoordT::ValueType;
6878 
6879  // All member data are mutable to allow for access methods to be const
6880 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY // 44 bytes total
6881  mutable CoordT mKey; // 3*4 = 12 bytes
6882 #else // 68 bytes total
6883  mutable CoordT mKeys[2]; // 2*3*4 = 24 bytes
6884 #endif
6885  mutable const RootT* mRoot;
6886  mutable const Node1T* mNode1;
6887  mutable const Node2T* mNode2;
6888 
6889 public:
6890  using BuildType = BuildT;
6891  using ValueType = ValueT;
6892  using CoordType = CoordT;
6893 
6894  static const int CacheLevels = 2;
6895 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
6896  using NodeInfo = typename ReadAccessor<ValueT, -1, -1, -1>::NodeInfo;
6897 #endif
6898  /// @brief Constructor from a root node
6900 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
6901  : mKey(CoordType::max())
6902 #else
6903  : mKeys{CoordType::max(), CoordType::max()}
6904 #endif
6905  , mRoot(&root)
6906  , mNode1(nullptr)
6907  , mNode2(nullptr)
6908  {
6909  }
6910 
6911  /// @brief Constructor from a grid
6913  : ReadAccessor(grid.tree().root())
6914  {
6915  }
6916 
6917  /// @brief Constructor from a tree
6919  : ReadAccessor(tree.root())
6920  {
6921  }
6922 
6923  /// @brief Reset this access to its initial state, i.e. with an empty cache
6925  {
6926 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
6927  mKey = CoordType::max();
6928 #else
6929  mKeys[0] = mKeys[1] = CoordType::max();
6930 #endif
6931  mNode1 = nullptr;
6932  mNode2 = nullptr;
6933  }
6934 
6935  __hostdev__ const RootT& root() const { return *mRoot; }
6936 
6937  /// @brief Defaults constructors
6938  ReadAccessor(const ReadAccessor&) = default;
6939  ~ReadAccessor() = default;
6940  ReadAccessor& operator=(const ReadAccessor&) = default;
6941 
6942 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
6943  __hostdev__ bool isCached1(CoordValueType dirty) const
6944  {
6945  if (!mNode1)
6946  return false;
6947  if (dirty & int32_t(~Node1T::MASK)) {
6948  mNode1 = nullptr;
6949  return false;
6950  }
6951  return true;
6952  }
6953  __hostdev__ bool isCached2(CoordValueType dirty) const
6954  {
6955  if (!mNode2)
6956  return false;
6957  if (dirty & int32_t(~Node2T::MASK)) {
6958  mNode2 = nullptr;
6959  return false;
6960  }
6961  return true;
6962  }
6963  __hostdev__ CoordValueType computeDirty(const CoordType& ijk) const
6964  {
6965  return (ijk[0] ^ mKey[0]) | (ijk[1] ^ mKey[1]) | (ijk[2] ^ mKey[2]);
6966  }
6967 #else
6968  __hostdev__ bool isCached1(const CoordType& ijk) const
6969  {
6970  return (ijk[0] & int32_t(~Node1T::MASK)) == mKeys[0][0] &&
6971  (ijk[1] & int32_t(~Node1T::MASK)) == mKeys[0][1] &&
6972  (ijk[2] & int32_t(~Node1T::MASK)) == mKeys[0][2];
6973  }
6974  __hostdev__ bool isCached2(const CoordType& ijk) const
6975  {
6976  return (ijk[0] & int32_t(~Node2T::MASK)) == mKeys[1][0] &&
6977  (ijk[1] & int32_t(~Node2T::MASK)) == mKeys[1][1] &&
6978  (ijk[2] & int32_t(~Node2T::MASK)) == mKeys[1][2];
6979  }
6980 #endif
6981 
6982 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
6984  {
6985  return this->template get<GetValue<BuildT>>(ijk);
6986  }
6987  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6988  __hostdev__ ValueType operator()(const CoordType& ijk) const { return this->template get<GetValue<BuildT>>(ijk); }
6989  __hostdev__ ValueType operator()(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
6990  __hostdev__ auto getNodeInfo(const CoordType& ijk) const { return this->template get<GetNodeInfo<BuildT>>(ijk); }
6991  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildT>>(ijk); }
6992  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildT>>(ijk, v); }
6993  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildT>>(ijk); }
6994 #else // NANOVDB_NEW_ACCESSOR_METHODS
6995 
6996  __hostdev__ ValueType getValue(const CoordType& ijk) const
6997  {
6998 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
6999  const CoordValueType dirty = this->computeDirty(ijk);
7000 #else
7001  auto&& dirty = ijk;
7002 #endif
7003  if (this->isCached1(dirty)) {
7004  return mNode1->getValueAndCache(ijk, *this);
7005  } else if (this->isCached2(dirty)) {
7006  return mNode2->getValueAndCache(ijk, *this);
7007  }
7008  return mRoot->getValueAndCache(ijk, *this);
7009  }
7010  __hostdev__ ValueType operator()(const CoordType& ijk) const
7011  {
7012  return this->getValue(ijk);
7013  }
7014  __hostdev__ ValueType operator()(int i, int j, int k) const
7015  {
7016  return this->getValue(CoordType(i, j, k));
7017  }
7018  __hostdev__ ValueType getValue(int i, int j, int k) const
7019  {
7020  return this->getValue(CoordType(i, j, k));
7021  }
7022  __hostdev__ NodeInfo getNodeInfo(const CoordType& ijk) const
7023  {
7024 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7025  const CoordValueType dirty = this->computeDirty(ijk);
7026 #else
7027  auto&& dirty = ijk;
7028 #endif
7029  if (this->isCached1(dirty)) {
7030  return mNode1->getNodeInfoAndCache(ijk, *this);
7031  } else if (this->isCached2(dirty)) {
7032  return mNode2->getNodeInfoAndCache(ijk, *this);
7033  }
7034  return mRoot->getNodeInfoAndCache(ijk, *this);
7035  }
7036 
7037  __hostdev__ bool isActive(const CoordType& ijk) const
7038  {
7039 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7040  const CoordValueType dirty = this->computeDirty(ijk);
7041 #else
7042  auto&& dirty = ijk;
7043 #endif
7044  if (this->isCached1(dirty)) {
7045  return mNode1->isActiveAndCache(ijk, *this);
7046  } else if (this->isCached2(dirty)) {
7047  return mNode2->isActiveAndCache(ijk, *this);
7048  }
7049  return mRoot->isActiveAndCache(ijk, *this);
7050  }
7051 
7052  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
7053  {
7054 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7055  const CoordValueType dirty = this->computeDirty(ijk);
7056 #else
7057  auto&& dirty = ijk;
7058 #endif
7059  if (this->isCached1(dirty)) {
7060  return mNode1->probeValueAndCache(ijk, v, *this);
7061  } else if (this->isCached2(dirty)) {
7062  return mNode2->probeValueAndCache(ijk, v, *this);
7063  }
7064  return mRoot->probeValueAndCache(ijk, v, *this);
7065  }
7066 
7067  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const
7068  {
7069 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7070  const CoordValueType dirty = this->computeDirty(ijk);
7071 #else
7072  auto&& dirty = ijk;
7073 #endif
7074  if (this->isCached1(dirty)) {
7075  return mNode1->probeLeafAndCache(ijk, *this);
7076  } else if (this->isCached2(dirty)) {
7077  return mNode2->probeLeafAndCache(ijk, *this);
7078  }
7079  return mRoot->probeLeafAndCache(ijk, *this);
7080  }
7081 #endif // NANOVDB_NEW_ACCESSOR_METHODS
7082 
7083  template<typename RayT>
7084  __hostdev__ uint32_t getDim(const CoordType& ijk, const RayT& ray) const
7085  {
7086 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7087  const CoordValueType dirty = this->computeDirty(ijk);
7088 #else
7089  auto&& dirty = ijk;
7090 #endif
7091  if (this->isCached1(dirty)) {
7092  return mNode1->getDimAndCache(ijk, ray, *this);
7093  } else if (this->isCached2(dirty)) {
7094  return mNode2->getDimAndCache(ijk, ray, *this);
7095  }
7096  return mRoot->getDimAndCache(ijk, ray, *this);
7097  }
7098 
7099  template<typename OpT, typename... ArgsT>
7100  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
7101  {
7102 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7103  const CoordValueType dirty = this->computeDirty(ijk);
7104 #else
7105  auto&& dirty = ijk;
7106 #endif
7107  if (this->isCached1(dirty)) {
7108  return mNode1->template getAndCache<OpT>(ijk, *this, args...);
7109  } else if (this->isCached2(dirty)) {
7110  return mNode2->template getAndCache<OpT>(ijk, *this, args...);
7111  }
7112  return mRoot->template getAndCache<OpT>(ijk, *this, args...);
7113  }
7114 
7115  template<typename OpT, typename... ArgsT>
7116  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args) const
7117  {
7118 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7119  const CoordValueType dirty = this->computeDirty(ijk);
7120 #else
7121  auto&& dirty = ijk;
7122 #endif
7123  if (this->isCached1(dirty)) {
7124  return const_cast<Node1T*>(mNode1)->template setAndCache<OpT>(ijk, *this, args...);
7125  } else if (this->isCached2(dirty)) {
7126  return const_cast<Node2T*>(mNode2)->template setAndCache<OpT>(ijk, *this, args...);
7127  }
7128  return const_cast<RootT*>(mRoot)->template setAndCache<OpT>(ijk, *this, args...);
7129  }
7130 
7131 private:
7132  /// @brief Allow nodes to insert themselves into the cache.
7133  template<typename>
7134  friend class RootNode;
7135  template<typename, uint32_t>
7136  friend class InternalNode;
7137  template<typename, typename, template<uint32_t> class, uint32_t>
7138  friend class LeafNode;
7139 
7140  /// @brief Inserts a leaf node and key pair into this ReadAccessor
7141  __hostdev__ void insert(const CoordType& ijk, const Node1T* node) const
7142  {
7143 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7144  mKey = ijk;
7145 #else
7146  mKeys[0] = ijk & ~Node1T::MASK;
7147 #endif
7148  mNode1 = node;
7149  }
7150  __hostdev__ void insert(const CoordType& ijk, const Node2T* node) const
7151  {
7152 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7153  mKey = ijk;
7154 #else
7155  mKeys[1] = ijk & ~Node2T::MASK;
7156 #endif
7157  mNode2 = node;
7158  }
7159  template<typename OtherNodeT>
7160  __hostdev__ void insert(const CoordType&, const OtherNodeT*) const {}
7161 }; // ReadAccessor<BuildT, LEVEL0, LEVEL1>
7162 
7163 /// @brief Node caching at all (three) tree levels
7164 template<typename BuildT>
7165 class ReadAccessor<BuildT, 0, 1, 2>
7166 {
7167  using GridT = NanoGrid<BuildT>; // grid
7168  using TreeT = NanoTree<BuildT>;
7169  using RootT = NanoRoot<BuildT>; // root node
7170  using NodeT2 = NanoUpper<BuildT>; // upper internal node
7171  using NodeT1 = NanoLower<BuildT>; // lower internal node
7172  using LeafT = NanoLeaf<BuildT>; // Leaf node
7173  using CoordT = typename RootT::CoordType;
7174  using ValueT = typename RootT::ValueType;
7175 
7176  using FloatType = typename RootT::FloatType;
7177  using CoordValueType = typename RootT::CoordT::ValueType;
7178 
7179  // All member data are mutable to allow for access methods to be const
7180 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY // 44 bytes total
7181  mutable CoordT mKey; // 3*4 = 12 bytes
7182 #else // 68 bytes total
7183  mutable CoordT mKeys[3]; // 3*3*4 = 36 bytes
7184 #endif
7185  mutable const RootT* mRoot;
7186  mutable const void* mNode[3]; // 4*8 = 32 bytes
7187 
7188 public:
7189  using BuildType = BuildT;
7190  using ValueType = ValueT;
7191  using CoordType = CoordT;
7192 
7193  static const int CacheLevels = 3;
7194 #ifndef NANOVDB_NEW_ACCESSOR_METHODS
7195  using NodeInfo = typename ReadAccessor<ValueT, -1, -1, -1>::NodeInfo;
7196 #endif
7197  /// @brief Constructor from a root node
7199 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7200  : mKey(CoordType::max())
7201 #else
7203 #endif
7204  , mRoot(&root)
7205  , mNode{nullptr, nullptr, nullptr}
7206  {
7207  }
7208 
7209  /// @brief Constructor from a grid
7210  __hostdev__ ReadAccessor(const GridT& grid)
7211  : ReadAccessor(grid.tree().root())
7212  {
7213  }
7214 
7215  /// @brief Constructor from a tree
7217  : ReadAccessor(tree.root())
7218  {
7219  }
7220 
7221  __hostdev__ const RootT& root() const { return *mRoot; }
7222 
7223  /// @brief Defaults constructors
7224  ReadAccessor(const ReadAccessor&) = default;
7225  ~ReadAccessor() = default;
7226  ReadAccessor& operator=(const ReadAccessor&) = default;
7227 
7228  /// @brief Return a const point to the cached node of the specified type
7229  ///
7230  /// @warning The return value could be NULL.
7231  template<typename NodeT>
7232  __hostdev__ const NodeT* getNode() const
7233  {
7234  using T = typename NodeTrait<TreeT, NodeT::LEVEL>::type;
7235  static_assert(is_same<T, NodeT>::value, "ReadAccessor::getNode: Invalid node type");
7236  return reinterpret_cast<const T*>(mNode[NodeT::LEVEL]);
7237  }
7238 
7239  template<int LEVEL>
7241  {
7242  using T = typename NodeTrait<TreeT, LEVEL>::type;
7243  static_assert(LEVEL >= 0 && LEVEL <= 2, "ReadAccessor::getNode: Invalid node type");
7244  return reinterpret_cast<const T*>(mNode[LEVEL]);
7245  }
7246 
7247  /// @brief Reset this access to its initial state, i.e. with an empty cache
7249  {
7250 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7251  mKey = CoordType::max();
7252 #else
7253  mKeys[0] = mKeys[1] = mKeys[2] = CoordType::max();
7254 #endif
7255  mNode[0] = mNode[1] = mNode[2] = nullptr;
7256  }
7257 
7258 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7259  template<typename NodeT>
7260  __hostdev__ bool isCached(CoordValueType dirty) const
7261  {
7262  if (!mNode[NodeT::LEVEL])
7263  return false;
7264  if (dirty & int32_t(~NodeT::MASK)) {
7265  mNode[NodeT::LEVEL] = nullptr;
7266  return false;
7267  }
7268  return true;
7269  }
7270 
7271  __hostdev__ CoordValueType computeDirty(const CoordType& ijk) const
7272  {
7273  return (ijk[0] ^ mKey[0]) | (ijk[1] ^ mKey[1]) | (ijk[2] ^ mKey[2]);
7274  }
7275 #else
7276  template<typename NodeT>
7277  __hostdev__ bool isCached(const CoordType& ijk) const
7278  {
7279  return (ijk[0] & int32_t(~NodeT::MASK)) == mKeys[NodeT::LEVEL][0] &&
7280  (ijk[1] & int32_t(~NodeT::MASK)) == mKeys[NodeT::LEVEL][1] &&
7281  (ijk[2] & int32_t(~NodeT::MASK)) == mKeys[NodeT::LEVEL][2];
7282  }
7283 #endif
7284 
7285 #ifdef NANOVDB_NEW_ACCESSOR_METHODS
7287  {
7288  return this->template get<GetValue<BuildT>>(ijk);
7289  }
7290  __hostdev__ ValueType getValue(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
7291  __hostdev__ ValueType operator()(const CoordType& ijk) const { return this->template get<GetValue<BuildT>>(ijk); }
7292  __hostdev__ ValueType operator()(int i, int j, int k) const { return this->template get<GetValue<BuildT>>(CoordType(i, j, k)); }
7293  __hostdev__ auto getNodeInfo(const CoordType& ijk) const { return this->template get<GetNodeInfo<BuildT>>(ijk); }
7294  __hostdev__ bool isActive(const CoordType& ijk) const { return this->template get<GetState<BuildT>>(ijk); }
7295  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const { return this->template get<ProbeValue<BuildT>>(ijk, v); }
7296  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const { return this->template get<GetLeaf<BuildT>>(ijk); }
7297 #else // NANOVDB_NEW_ACCESSOR_METHODS
7298 
7299  __hostdev__ ValueType getValue(const CoordType& ijk) const
7300  {
7301 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7302  const CoordValueType dirty = this->computeDirty(ijk);
7303 #else
7304  auto&& dirty = ijk;
7305 #endif
7306  if (this->isCached<LeafT>(dirty)) {
7307  return ((LeafT*)mNode[0])->getValue(ijk);
7308  } else if (this->isCached<NodeT1>(dirty)) {
7309  return ((NodeT1*)mNode[1])->getValueAndCache(ijk, *this);
7310  } else if (this->isCached<NodeT2>(dirty)) {
7311  return ((NodeT2*)mNode[2])->getValueAndCache(ijk, *this);
7312  }
7313  return mRoot->getValueAndCache(ijk, *this);
7314  }
7315  __hostdev__ ValueType operator()(const CoordType& ijk) const
7316  {
7317  return this->getValue(ijk);
7318  }
7319  __hostdev__ ValueType operator()(int i, int j, int k) const
7320  {
7321  return this->getValue(CoordType(i, j, k));
7322  }
7323  __hostdev__ ValueType getValue(int i, int j, int k) const
7324  {
7325  return this->getValue(CoordType(i, j, k));
7326  }
7327 
7328  __hostdev__ NodeInfo getNodeInfo(const CoordType& ijk) const
7329  {
7330 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7331  const CoordValueType dirty = this->computeDirty(ijk);
7332 #else
7333  auto&& dirty = ijk;
7334 #endif
7335  if (this->isCached<LeafT>(dirty)) {
7336  return ((LeafT*)mNode[0])->getNodeInfoAndCache(ijk, *this);
7337  } else if (this->isCached<NodeT1>(dirty)) {
7338  return ((NodeT1*)mNode[1])->getNodeInfoAndCache(ijk, *this);
7339  } else if (this->isCached<NodeT2>(dirty)) {
7340  return ((NodeT2*)mNode[2])->getNodeInfoAndCache(ijk, *this);
7341  }
7342  return mRoot->getNodeInfoAndCache(ijk, *this);
7343  }
7344 
7345  __hostdev__ bool isActive(const CoordType& ijk) const
7346  {
7347 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7348  const CoordValueType dirty = this->computeDirty(ijk);
7349 #else
7350  auto&& dirty = ijk;
7351 #endif
7352  if (this->isCached<LeafT>(dirty)) {
7353  return ((LeafT*)mNode[0])->isActive(ijk);
7354  } else if (this->isCached<NodeT1>(dirty)) {
7355  return ((NodeT1*)mNode[1])->isActiveAndCache(ijk, *this);
7356  } else if (this->isCached<NodeT2>(dirty)) {
7357  return ((NodeT2*)mNode[2])->isActiveAndCache(ijk, *this);
7358  }
7359  return mRoot->isActiveAndCache(ijk, *this);
7360  }
7361 
7362  __hostdev__ bool probeValue(const CoordType& ijk, ValueType& v) const
7363  {
7364 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7365  const CoordValueType dirty = this->computeDirty(ijk);
7366 #else
7367  auto&& dirty = ijk;
7368 #endif
7369  if (this->isCached<LeafT>(dirty)) {
7370  return ((LeafT*)mNode[0])->probeValue(ijk, v);
7371  } else if (this->isCached<NodeT1>(dirty)) {
7372  return ((NodeT1*)mNode[1])->probeValueAndCache(ijk, v, *this);
7373  } else if (this->isCached<NodeT2>(dirty)) {
7374  return ((NodeT2*)mNode[2])->probeValueAndCache(ijk, v, *this);
7375  }
7376  return mRoot->probeValueAndCache(ijk, v, *this);
7377  }
7378  __hostdev__ const LeafT* probeLeaf(const CoordType& ijk) const
7379  {
7380 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7381  const CoordValueType dirty = this->computeDirty(ijk);
7382 #else
7383  auto&& dirty = ijk;
7384 #endif
7385  if (this->isCached<LeafT>(dirty)) {
7386  return ((LeafT*)mNode[0]);
7387  } else if (this->isCached<NodeT1>(dirty)) {
7388  return ((NodeT1*)mNode[1])->probeLeafAndCache(ijk, *this);
7389  } else if (this->isCached<NodeT2>(dirty)) {
7390  return ((NodeT2*)mNode[2])->probeLeafAndCache(ijk, *this);
7391  }
7392  return mRoot->probeLeafAndCache(ijk, *this);
7393  }
7394 #endif // NANOVDB_NEW_ACCESSOR_METHODS
7395 
7396  template<typename OpT, typename... ArgsT>
7397  __hostdev__ auto get(const CoordType& ijk, ArgsT&&... args) const
7398  {
7399 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7400  const CoordValueType dirty = this->computeDirty(ijk);
7401 #else
7402  auto&& dirty = ijk;
7403 #endif
7404  if (this->isCached<LeafT>(dirty)) {
7405  return ((const LeafT*)mNode[0])->template getAndCache<OpT>(ijk, *this, args...);
7406  } else if (this->isCached<NodeT1>(dirty)) {
7407  return ((const NodeT1*)mNode[1])->template getAndCache<OpT>(ijk, *this, args...);
7408  } else if (this->isCached<NodeT2>(dirty)) {
7409  return ((const NodeT2*)mNode[2])->template getAndCache<OpT>(ijk, *this, args...);
7410  }
7411  return mRoot->template getAndCache<OpT>(ijk, *this, args...);
7412  }
7413 
7414  template<typename OpT, typename... ArgsT>
7415  __hostdev__ auto set(const CoordType& ijk, ArgsT&&... args) const
7416  {
7417 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7418  const CoordValueType dirty = this->computeDirty(ijk);
7419 #else
7420  auto&& dirty = ijk;
7421 #endif
7422  if (this->isCached<LeafT>(dirty)) {
7423  return ((LeafT*)mNode[0])->template setAndCache<OpT>(ijk, *this, args...);
7424  } else if (this->isCached<NodeT1>(dirty)) {
7425  return ((NodeT1*)mNode[1])->template setAndCache<OpT>(ijk, *this, args...);
7426  } else if (this->isCached<NodeT2>(dirty)) {
7427  return ((NodeT2*)mNode[2])->template setAndCache<OpT>(ijk, *this, args...);
7428  }
7429  return ((RootT*)mRoot)->template setAndCache<OpT>(ijk, *this, args...);
7430  }
7431 
7432  template<typename RayT>
7433  __hostdev__ uint32_t getDim(const CoordType& ijk, const RayT& ray) const
7434  {
7435 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7436  const CoordValueType dirty = this->computeDirty(ijk);
7437 #else
7438  auto&& dirty = ijk;
7439 #endif
7440  if (this->isCached<LeafT>(dirty)) {
7441  return ((LeafT*)mNode[0])->getDimAndCache(ijk, ray, *this);
7442  } else if (this->isCached<NodeT1>(dirty)) {
7443  return ((NodeT1*)mNode[1])->getDimAndCache(ijk, ray, *this);
7444  } else if (this->isCached<NodeT2>(dirty)) {
7445  return ((NodeT2*)mNode[2])->getDimAndCache(ijk, ray, *this);
7446  }
7447  return mRoot->getDimAndCache(ijk, ray, *this);
7448  }
7449 
7450 private:
7451  /// @brief Allow nodes to insert themselves into the cache.
7452  template<typename>
7453  friend class RootNode;
7454  template<typename, uint32_t>
7455  friend class InternalNode;
7456  template<typename, typename, template<uint32_t> class, uint32_t>
7457  friend class LeafNode;
7458 
7459  /// @brief Inserts a leaf node and key pair into this ReadAccessor
7460  template<typename NodeT>
7461  __hostdev__ void insert(const CoordType& ijk, const NodeT* node) const
7462  {
7463 #ifdef NANOVDB_USE_SINGLE_ACCESSOR_KEY
7464  mKey = ijk;
7465 #else
7466  mKeys[NodeT::LEVEL] = ijk & ~NodeT::MASK;
7467 #endif
7468  mNode[NodeT::LEVEL] = node;
7469  }
7470 }; // ReadAccessor<BuildT, 0, 1, 2>
7471 
7472 //////////////////////////////////////////////////
7473 
7474 /// @brief Free-standing function for convenient creation of a ReadAccessor with
7475 /// optional and customizable node caching.
7476 ///
7477 /// @details createAccessor<>(grid): No caching of nodes and hence it's thread-safe but slow
7478 /// createAccessor<0>(grid): Caching of leaf nodes only
7479 /// createAccessor<1>(grid): Caching of lower internal nodes only
7480 /// createAccessor<2>(grid): Caching of upper internal nodes only
7481 /// createAccessor<0,1>(grid): Caching of leaf and lower internal nodes
7482 /// createAccessor<0,2>(grid): Caching of leaf and upper internal nodes
7483 /// createAccessor<1,2>(grid): Caching of lower and upper internal nodes
7484 /// createAccessor<0,1,2>(grid): Caching of all nodes at all tree levels
7485 
7486 template<int LEVEL0 = -1, int LEVEL1 = -1, int LEVEL2 = -1, typename ValueT = float>
7488 {
7490 }
7491 
7492 template<int LEVEL0 = -1, int LEVEL1 = -1, int LEVEL2 = -1, typename ValueT = float>
7494 {
7496 }
7497 
7498 template<int LEVEL0 = -1, int LEVEL1 = -1, int LEVEL2 = -1, typename ValueT = float>
7500 {
7502 }
7503 
7504 //////////////////////////////////////////////////
7505 
7506 /// @brief This is a convenient class that allows for access to grid meta-data
7507 /// that are independent of the value type of a grid. That is, this class
7508 /// can be used to get information about a grid without actually knowing
7509 /// its ValueType.
7511 { // 768 bytes (32 byte aligned)
7512  GridData mGridData; // 672B
7513  TreeData mTreeData; // 64B
7514  CoordBBox mIndexBBox; // 24B. AABB of active values in index space.
7515  uint32_t mRootTableSize, mPadding{0}; // 8B
7516 
7517 public:
7518  template<typename T>
7520  {
7521  mGridData = *grid.data();
7522  mTreeData = *grid.tree().data();
7523  mIndexBBox = grid.indexBBox();
7524  mRootTableSize = grid.tree().root().getTableSize();
7525  }
7526  GridMetaData(const GridData* gridData)
7527  {
7528  static_assert(8 * 96 == sizeof(GridMetaData), "GridMetaData has unexpected size");
7529  if (GridMetaData::safeCast(gridData)) {
7530  memcpy64(this, gridData, 96);
7531  } else {// otherwise copy each member individually
7532  mGridData = *gridData;
7533  mTreeData = *reinterpret_cast<const TreeData*>(gridData->treePtr());
7534  mIndexBBox = gridData->indexBBox();
7535  mRootTableSize = gridData->rootTableSize();
7536  }
7537  }
7538  /// @brief return true if the RootData follows right after the TreeData.
7539  /// If so, this implies that it's safe to cast the grid from which
7540  /// this instance was constructed to a GridMetaData
7541  __hostdev__ bool safeCast() const { return mTreeData.isRootNext(); }
7542 
7543  /// @brief return true if it is safe to cast the grid to a pointer
7544  /// of type GridMetaData, i.e. construction can be avoided.
7545  __hostdev__ static bool safeCast(const GridData *gridData){
7546  NANOVDB_ASSERT(gridData && gridData->isValid());
7547  return gridData->isRootConnected();
7548  }
7549  /// @brief return true if it is safe to cast the grid to a pointer
7550  /// of type GridMetaData, i.e. construction can be avoided.
7551  template<typename T>
7552  __hostdev__ static bool safeCast(const NanoGrid<T>& grid){return grid.tree().isRootNext();}
7553  __hostdev__ bool isValid() const { return mGridData.isValid(); }
7554  __hostdev__ const GridType& gridType() const { return mGridData.mGridType; }
7555  __hostdev__ const GridClass& gridClass() const { return mGridData.mGridClass; }
7556  __hostdev__ bool isLevelSet() const { return mGridData.mGridClass == GridClass::LevelSet; }
7557  __hostdev__ bool isFogVolume() const { return mGridData.mGridClass == GridClass::FogVolume; }
7558  __hostdev__ bool isStaggered() const { return mGridData.mGridClass == GridClass::Staggered; }
7559  __hostdev__ bool isPointIndex() const { return mGridData.mGridClass == GridClass::PointIndex; }
7560  __hostdev__ bool isGridIndex() const { return mGridData.mGridClass == GridClass::IndexGrid; }
7561  __hostdev__ bool isPointData() const { return mGridData.mGridClass == GridClass::PointData; }
7562  __hostdev__ bool isMask() const { return mGridData.mGridClass == GridClass::Topology; }
7563  __hostdev__ bool isUnknown() const { return mGridData.mGridClass == GridClass::Unknown; }
7564  __hostdev__ bool hasMinMax() const { return mGridData.mFlags.isMaskOn(GridFlags::HasMinMax); }
7565  __hostdev__ bool hasBBox() const { return mGridData.mFlags.isMaskOn(GridFlags::HasBBox); }
7567  __hostdev__ bool hasAverage() const { return mGridData.mFlags.isMaskOn(GridFlags::HasAverage); }
7570  __hostdev__ uint64_t gridSize() const { return mGridData.mGridSize; }
7571  __hostdev__ uint32_t gridIndex() const { return mGridData.mGridIndex; }
7572  __hostdev__ uint32_t gridCount() const { return mGridData.mGridCount; }
7573  __hostdev__ const char* shortGridName() const { return mGridData.mGridName; }
7574  __hostdev__ const Map& map() const { return mGridData.mMap; }
7575  __hostdev__ const BBox<Vec3d>& worldBBox() const { return mGridData.mWorldBBox; }
7576  __hostdev__ const BBox<Coord>& indexBBox() const { return mIndexBBox; }
7577  __hostdev__ Vec3d voxelSize() const { return mGridData.mVoxelSize; }
7578  __hostdev__ int blindDataCount() const { return mGridData.mBlindMetadataCount; }
7579  __hostdev__ uint64_t activeVoxelCount() const { return mTreeData.mVoxelCount; }
7580  __hostdev__ const uint32_t& activeTileCount(uint32_t level) const { return mTreeData.mTileCount[level - 1]; }
7581  __hostdev__ uint32_t nodeCount(uint32_t level) const { return mTreeData.mNodeCount[level]; }
7582  __hostdev__ uint64_t checksum() const { return mGridData.mChecksum; }
7583  __hostdev__ uint32_t rootTableSize() const { return mRootTableSize; }
7584  __hostdev__ bool isEmpty() const { return mRootTableSize == 0; }
7585  __hostdev__ Version version() const { return mGridData.mVersion; }
7586 }; // GridMetaData
7587 
7588 /// @brief Class to access points at a specific voxel location
7589 ///
7590 /// @note If GridClass::PointIndex AttT should be uint32_t and if GridClass::PointData Vec3f
7591 template<typename AttT, typename BuildT = uint32_t>
7592 class PointAccessor : public DefaultReadAccessor<BuildT>
7593 {
7594  using AccT = DefaultReadAccessor<BuildT>;
7595  const NanoGrid<BuildT>& mGrid;
7596  const AttT* mData;
7597 
7598 public:
7600  : AccT(grid.tree().root())
7601  , mGrid(grid)
7602  , mData(grid.template getBlindData<AttT>(0))
7603  {
7604  NANOVDB_ASSERT(grid.gridType() == mapToGridType<BuildT>());
7607  }
7608 
7609  /// @brief return true if this access was initialized correctly
7610  __hostdev__ operator bool() const { return mData != nullptr; }
7611 
7612  __hostdev__ const NanoGrid<BuildT>& grid() const { return mGrid; }
7613 
7614  /// @brief Return the total number of point in the grid and set the
7615  /// iterators to the complete range of points.
7616  __hostdev__ uint64_t gridPoints(const AttT*& begin, const AttT*& end) const
7617  {
7618  const uint64_t count = mGrid.blindMetaData(0u).mValueCount;
7619  begin = mData;
7620  end = begin + count;
7621  return count;
7622  }
7623  /// @brief Return the number of points in the leaf node containing the coordinate @a ijk.
7624  /// If this return value is larger than zero then the iterators @a begin and @a end
7625  /// will point to all the attributes contained within that leaf node.
7626  __hostdev__ uint64_t leafPoints(const Coord& ijk, const AttT*& begin, const AttT*& end) const
7627  {
7628  auto* leaf = this->probeLeaf(ijk);
7629  if (leaf == nullptr) {
7630  return 0;
7631  }
7632  begin = mData + leaf->minimum();
7633  end = begin + leaf->maximum();
7634  return leaf->maximum();
7635  }
7636 
7637  /// @brief get iterators over attributes to points at a specific voxel location
7638  __hostdev__ uint64_t voxelPoints(const Coord& ijk, const AttT*& begin, const AttT*& end) const
7639  {
7640  begin = end = nullptr;
7641  if (auto* leaf = this->probeLeaf(ijk)) {
7642  const uint32_t offset = NanoLeaf<BuildT>::CoordToOffset(ijk);
7643  if (leaf->isActive(offset)) {
7644  begin = mData + leaf->minimum();
7645  end = begin + leaf->getValue(offset);
7646  if (offset > 0u)
7647  begin += leaf->getValue(offset - 1);
7648  }
7649  }
7650  return end - begin;
7651  }
7652 }; // PointAccessor
7653 
7654 template<typename AttT>
7655 class PointAccessor<AttT, Point> : public DefaultReadAccessor<Point>
7656 {
7657  using AccT = DefaultReadAccessor<Point>;
7658  const NanoGrid<Point>& mGrid;
7659  const AttT* mData;
7660 
7661 public:
7663  : AccT(grid.tree().root())
7664  , mGrid(grid)
7665  , mData(grid.template getBlindData<AttT>(0))
7666  {
7667  NANOVDB_ASSERT(mData);
7674  }
7675 
7676  /// @brief return true if this access was initialized correctly
7677  __hostdev__ operator bool() const { return mData != nullptr; }
7678 
7679  __hostdev__ const NanoGrid<Point>& grid() const { return mGrid; }
7680 
7681  /// @brief Return the total number of point in the grid and set the
7682  /// iterators to the complete range of points.
7683  __hostdev__ uint64_t gridPoints(const AttT*& begin, const AttT*& end) const
7684  {
7685  const uint64_t count = mGrid.blindMetaData(0u).mValueCount;
7686  begin = mData;
7687  end = begin + count;
7688  return count;
7689  }
7690  /// @brief Return the number of points in the leaf node containing the coordinate @a ijk.
7691  /// If this return value is larger than zero then the iterators @a begin and @a end
7692  /// will point to all the attributes contained within that leaf node.
7693  __hostdev__ uint64_t leafPoints(const Coord& ijk, const AttT*& begin, const AttT*& end) const
7694  {
7695  auto* leaf = this->probeLeaf(ijk);
7696  if (leaf == nullptr)
7697  return 0;
7698  begin = mData + leaf->offset();
7699  end = begin + leaf->pointCount();
7700  return leaf->pointCount();
7701  }
7702 
7703  /// @brief get iterators over attributes to points at a specific voxel location
7704  __hostdev__ uint64_t voxelPoints(const Coord& ijk, const AttT*& begin, const AttT*& end) const
7705  {
7706  if (auto* leaf = this->probeLeaf(ijk)) {
7707  const uint32_t n = NanoLeaf<Point>::CoordToOffset(ijk);
7708  if (leaf->isActive(n)) {
7709  begin = mData + leaf->first(n);
7710  end = mData + leaf->last(n);
7711  return end - begin;
7712  }
7713  }
7714  begin = end = nullptr;
7715  return 0u; // no leaf or inactive voxel
7716  }
7717 }; // PointAccessor<AttT, Point>
7718 
7719 /// @brief Class to access values in channels at a specific voxel location.
7720 ///
7721 /// @note The ChannelT template parameter can be either const and non-const.
7722 template<typename ChannelT, typename IndexT = ValueIndex>
7723 class ChannelAccessor : public DefaultReadAccessor<IndexT>
7724 {
7725  static_assert(BuildTraits<IndexT>::is_index, "Expected an index build type");
7727 
7728  const NanoGrid<IndexT>& mGrid;
7729  ChannelT* mChannel;
7730 
7731 public:
7732  using ValueType = ChannelT;
7735 
7736  /// @brief Ctor from an IndexGrid and an integer ID of an internal channel
7737  /// that is assumed to exist as blind data in the IndexGrid.
7738  __hostdev__ ChannelAccessor(const NanoGrid<IndexT>& grid, uint32_t channelID = 0u)
7739  : BaseT(grid.tree().root())
7740  , mGrid(grid)
7741  , mChannel(nullptr)
7742  {
7743  NANOVDB_ASSERT(isIndex(grid.gridType()));
7745  this->setChannel(channelID);
7746  }
7747 
7748  /// @brief Ctor from an IndexGrid and an external channel
7749  __hostdev__ ChannelAccessor(const NanoGrid<IndexT>& grid, ChannelT* channelPtr)
7750  : BaseT(grid.tree().root())
7751  , mGrid(grid)
7752  , mChannel(channelPtr)
7753  {
7754  NANOVDB_ASSERT(isIndex(grid.gridType()));
7756  }
7757 
7758  /// @brief return true if this access was initialized correctly
7759  __hostdev__ operator bool() const { return mChannel != nullptr; }
7760 
7761  /// @brief Return a const reference to the IndexGrid
7762  __hostdev__ const NanoGrid<IndexT>& grid() const { return mGrid; }
7763 
7764  /// @brief Return a const reference to the tree of the IndexGrid
7765  __hostdev__ const TreeType& tree() const { return mGrid.tree(); }
7766 
7767  /// @brief Return a vector of the axial voxel sizes
7768  __hostdev__ const Vec3d& voxelSize() const { return mGrid.voxelSize(); }
7769 
7770  /// @brief Return total number of values indexed by the IndexGrid
7771  __hostdev__ const uint64_t& valueCount() const { return mGrid.valueCount(); }
7772 
7773  /// @brief Change to an external channel
7774  /// @return Pointer to channel data
7775  __hostdev__ ChannelT* setChannel(ChannelT* channelPtr) {return mChannel = channelPtr;}
7776 
7777  /// @brief Change to an internal channel, assuming it exists as as blind data
7778  /// in the IndexGrid.
7779  /// @return Pointer to channel data, which could be NULL if channelID is out of range or
7780  /// if ChannelT does not match the value type of the blind data
7781  __hostdev__ ChannelT* setChannel(uint32_t channelID)
7782  {
7783  return mChannel = const_cast<ChannelT*>(mGrid.template getBlindData<ChannelT>(channelID));
7784  }
7785 
7786  /// @brief Return the linear offset into a channel that maps to the specified coordinate
7787  __hostdev__ uint64_t getIndex(const Coord& ijk) const { return BaseT::getValue(ijk); }
7788  __hostdev__ uint64_t idx(int i, int j, int k) const { return BaseT::getValue(Coord(i, j, k)); }
7789 
7790  /// @brief Return the value from a cached channel that maps to the specified coordinate
7791  __hostdev__ ChannelT& getValue(const Coord& ijk) const { return mChannel[BaseT::getValue(ijk)]; }
7792  __hostdev__ ChannelT& operator()(const Coord& ijk) const { return this->getValue(ijk); }
7793  __hostdev__ ChannelT& operator()(int i, int j, int k) const { return this->getValue(Coord(i, j, k)); }
7794 
7795  /// @brief return the state and updates the value of the specified voxel
7796  __hostdev__ bool probeValue(const Coord& ijk, typename remove_const<ChannelT>::type& v) const
7797  {
7798  uint64_t idx;
7799  const bool isActive = BaseT::probeValue(ijk, idx);
7800  v = mChannel[idx];
7801  return isActive;
7802  }
7803  /// @brief Return the value from a specified channel that maps to the specified coordinate
7804  ///
7805  /// @note The template parameter can be either const or non-const
7806  template<typename T>
7807  __hostdev__ T& getValue(const Coord& ijk, T* channelPtr) const { return channelPtr[BaseT::getValue(ijk)]; }
7808 
7809 }; // ChannelAccessor
7810 
7811 #if 0
7812 // This MiniGridHandle class is only included as a stand-alone example. Note that aligned_alloc is a C++17 feature!
7813 // Normally we recommend using GridHandle defined in util/GridHandle.h but this minimal implementation could be an
7814 // alternative when using the IO medthods defined below.
7815 struct MiniGridHandle {
7816  struct BufferType {
7817  uint8_t *data;
7818  uint64_t size;
7819  BufferType(uint64_t n=0) : data(std::aligned_alloc(NANOVDB_DATA_ALIGNMENT, n)), size(n) {assert(isValid(data));}
7820  BufferType(BufferType &&other) : data(other.data), size(other.size) {other.data=nullptr; other.size=0;}
7821  ~BufferType() {std::free(data);}
7822  BufferType& operator=(const BufferType &other) = delete;
7823  BufferType& operator=(BufferType &&other){data=other.data; size=other.size; other.data=nullptr; other.size=0; return *this;}
7824  static BufferType create(size_t n, BufferType* dummy = nullptr) {return BufferType(n);}
7825  } buffer;
7826  MiniGridHandle(BufferType &&buf) : buffer(std::move(buf)) {}
7827  const uint8_t* data() const {return buffer.data;}
7828 };// MiniGridHandle
7829 #endif
7830 
7831 namespace io {
7832 
7833 /// @brief Define compression codecs
7834 ///
7835 /// @note NONE is the default, ZIP is slow but compact and BLOSC offers a great balance.
7836 ///
7837 /// @throw NanoVDB optionally supports ZIP and BLOSC compression and will throw an exception
7838 /// if its support is required but missing.
7839 enum class Codec : uint16_t { NONE = 0,
7840  ZIP = 1,
7841  BLOSC = 2,
7842  END = 3 };
7843 
7844 /// @brief Data encoded at the head of each segment of a file or stream.
7845 ///
7846 /// @note A file or stream is composed of one or more segments that each contain
7847 // one or more grids.
7848 struct FileHeader {// 16 bytes
7849  uint64_t magic;// 8 bytes
7850  Version version;// 4 bytes version numbers
7851  uint16_t gridCount;// 2 bytes
7852  Codec codec;// 2 bytes
7853  bool isValid() const {return magic == NANOVDB_MAGIC_NUMBER || magic == NANOVDB_MAGIC_FILE;}
7854 }; // FileHeader ( 16 bytes = 2 words )
7855 
7856 // @brief Data encoded for each of the grids associated with a segment.
7857 // Grid size in memory (uint64_t) |
7858 // Grid size on disk (uint64_t) |
7859 // Grid name hash key (uint64_t) |
7860 // Numer of active voxels (uint64_t) |
7861 // Grid type (uint32_t) |
7862 // Grid class (uint32_t) |
7863 // Characters in grid name (uint32_t) |
7864 // AABB in world space (2*3*double) | one per grid in file
7865 // AABB in index space (2*3*int) |
7866 // Size of a voxel in world units (3*double) |
7867 // Byte size of the grid name (uint32_t) |
7868 // Number of nodes per level (4*uint32_t) |
7869 // Numer of active tiles per level (3*uint32_t) |
7870 // Codec for file compression (uint16_t) |
7871 // Padding due to 8B alignment (uint16_t) |
7872 // Version number (uint32_t) |
7874 {// 176 bytes
7875  uint64_t gridSize, fileSize, nameKey, voxelCount; // 4 * 8 = 32B.
7878  BBox<Vec3d> worldBBox; // 2 * 3 * 8 = 48B.
7879  CoordBBox indexBBox; // 2 * 3 * 4 = 24B.
7880  Vec3d voxelSize; // 24B.
7881  uint32_t nameSize; // 4B.
7882  uint32_t nodeCount[4]; //4 x 4 = 16B
7883  uint32_t tileCount[3];// 3 x 4 = 12B
7884  Codec codec; // 2B
7885  uint16_t padding;// 2B, due to 8B alignment from uint64_t
7887 }; // FileMetaData
7888 
7889 // the following code block uses std and therefore needs to be ignored by CUDA and HIP
7890 #if !defined(__CUDA_ARCH__) && !defined(__HIP__)
7891 
7892 inline const char* toStr(Codec codec)
7893 {
7894  static const char * LUT[] = { "NONE", "ZIP", "BLOSC" , "END" };
7895  static_assert(sizeof(LUT) / sizeof(char*) - 1 == int(Codec::END), "Unexpected size of LUT");
7896  return LUT[static_cast<int>(codec)];
7897 }
7898 
7899 // Note that starting with version 32.6.0 it is possible to write and read raw grid buffers to
7900 // files, e.g. os.write((const char*)&buffer.data(), buffer.size()) or more conveniently as
7901 // handle.write(fileName). In addition to this simple approach we offer the methods below to
7902 // write traditional uncompressed nanovdb files that unlike raw files include metadata that
7903 // is used for tools like nanovdb_print.
7904 
7905 ///
7906 /// @brief This is a standalone alternative to io::writeGrid(...,Codec::NONE) defined in util/IO.h
7907 /// Unlike the latter this function has no dependencies at all, not even NanoVDB.h, so it also
7908 /// works if client code only includes PNanoVDB.h!
7909 ///
7910 /// @details Writes a raw NanoVDB buffer, possibly with multiple grids, to a stream WITHOUT compression.
7911 /// It follows all the conventions in util/IO.h so the stream can be read by all existing client
7912 /// code of NanoVDB.
7913 ///
7914 /// @note This method will always write uncompressed grids to the stream, i.e. Blosc or ZIP compression
7915 /// is never applied! This is a fundamental limitation and feature of this standalone function.
7916 ///
7917 /// @throw std::invalid_argument if buffer does not point to a valid NanoVDB grid.
7918 ///
7919 /// @warning This is pretty ugly code that involves lots of pointer and bit manipulations - not for the faint of heart :)
7920 template<typename StreamT> // StreamT class must support: "void write(const char*, size_t)"
7921 void writeUncompressedGrid(StreamT& os, const GridData* gridData, bool raw = false)
7922 {
7923  NANOVDB_ASSERT(gridData->mMagic == NANOVDB_MAGIC_NUMBER || gridData->mMagic == NANOVDB_MAGIC_GRID);
7924  NANOVDB_ASSERT(gridData->mVersion.isCompatible());
7925  if (!raw) {// segment with a single grid: FileHeader, FileMetaData, gridName, Grid
7926 #ifdef NANOVDB_USE_NEW_MAGIC_NUMBERS
7927  FileHeader head{NANOVDB_MAGIC_FILE, gridData->mVersion, 1u, Codec::NONE};
7928 #else
7929  FileHeader head{NANOVDB_MAGIC_NUMBER, gridData->mVersion, 1u, Codec::NONE};
7930 #endif
7931  const char* gridName = gridData->gridName();
7932  uint32_t nameSize = 1; // '\0'
7933  for (const char* p = gridName; *p != '\0'; ++p) ++nameSize;
7934  const TreeData* treeData = (const TreeData*)gridData->treePtr();
7935  FileMetaData meta{gridData->mGridSize, gridData->mGridSize, 0u, treeData->mVoxelCount,
7936  gridData->mGridType, gridData->mGridClass, gridData->mWorldBBox,
7937  treeData->bbox(), gridData->mVoxelSize, nameSize,
7938  {treeData->mNodeCount[0], treeData->mNodeCount[1], treeData->mNodeCount[2], 1u},
7939  {treeData->mTileCount[0], treeData->mTileCount[1], treeData->mTileCount[2]},
7940  Codec::NONE, 0u, gridData->mVersion }; // FileMetaData
7941  os.write((const char*)&head, sizeof(FileHeader)); // write header
7942  os.write((const char*)&meta, sizeof(FileMetaData)); // write meta data
7943  os.write(gridName, nameSize); // write grid name
7944  }
7945  os.write((const char*)gridData, gridData->mGridSize);// write the grid
7946 }// writeUncompressedGrid
7947 
7948 /// @brief write multiple NanoVDB grids to a single file, without compression.
7949 /// @note To write all grids in a single GridHandle simply use handle.write("fieNane")
7950 template<typename GridHandleT, template<typename...> class VecT>
7951 void writeUncompressedGrids(const char* fileName, const VecT<GridHandleT>& handles, bool raw = false)
7952 {
7953 #ifdef NANOVDB_USE_IOSTREAMS // use this to switch between std::ofstream or FILE implementations
7954  std::ofstream os(fileName, std::ios::out | std::ios::binary | std::ios::trunc);
7955 #else
7956  struct StreamT {
7957  FILE* fptr;
7958  StreamT(const char* name) { fptr = fopen(name, "wb"); }
7959  ~StreamT() { fclose(fptr); }
7960  void write(const char* data, size_t n) { fwrite(data, 1, n, fptr); }
7961  bool is_open() const { return fptr != NULL; }
7962  } os(fileName);
7963 #endif
7964  if (!os.is_open()) {
7965  fprintf(stderr, "nanovdb::writeUncompressedGrids: Unable to open file \"%s\"for output\n", fileName);
7966  exit(EXIT_FAILURE);
7967  }
7968  for (auto& h : handles) {
7969  for (uint32_t n=0; n<h.gridCount(); ++n) writeUncompressedGrid(os, h.gridData(n), raw);
7970  }
7971 } // writeUncompressedGrids
7972 
7973 /// @brief read all uncompressed grids from a stream and return their handles.
7974 ///
7975 /// @throw std::invalid_argument if stream does not contain a single uncompressed valid NanoVDB grid
7976 ///
7977 /// @details StreamT class must support: "bool read(char*, size_t)" and "void skip(uint32_t)"
7978 template<typename GridHandleT, typename StreamT, template<typename...> class VecT>
7979 VecT<GridHandleT> readUncompressedGrids(StreamT& is, const typename GridHandleT::BufferType& pool = typename GridHandleT::BufferType())
7980 {
7981  VecT<GridHandleT> handles;
7982  GridData data;
7983  is.read((char*)&data, 40);// we only need to load the first 40 bytes
7984  if (data.mMagic == NANOVDB_MAGIC_GRID || data.isValid()) {// stream contains a raw grid buffer
7985  uint64_t size = data.mGridSize, sum = 0u;
7986  while(data.mGridIndex + 1u < data.mGridCount) {
7987  is.skip(data.mGridSize - 40);// skip grid
7988  is.read((char*)&data, 40);// read 40 bytes
7989  sum += data.mGridSize;
7990  }
7991  is.skip(-int64_t(sum + 40));// rewind to start
7992  auto buffer = GridHandleT::BufferType::create(size + sum, &pool);
7993  is.read((char*)(buffer.data()), buffer.size());
7994  handles.emplace_back(std::move(buffer));
7995  } else {// Header0, MetaData0, gridName0, Grid0...HeaderN, MetaDataN, gridNameN, GridN
7996  is.skip(-40);// rewind
7997  FileHeader head;
7998  while(is.read((char*)&head, sizeof(FileHeader))) {
7999  if (!head.isValid()) {
8000  fprintf(stderr, "nanovdb::readUncompressedGrids: invalid magic number = \"%s\"\n", (const char*)&(head.magic));
8001  exit(EXIT_FAILURE);
8002  } else if (!head.version.isCompatible()) {
8003  fprintf(stderr, "nanovdb::readUncompressedGrids: invalid major version = \"%s\"\n", head.version.c_str());
8004  exit(EXIT_FAILURE);
8005  } else if (head.codec != Codec::NONE) {
8006  fprintf(stderr, "nanovdb::readUncompressedGrids: invalid codec = \"%s\"\n", toStr(head.codec));
8007  exit(EXIT_FAILURE);
8008  }
8009  FileMetaData meta;
8010  for (uint16_t i = 0; i < head.gridCount; ++i) { // read all grids in segment
8011  is.read((char*)&meta, sizeof(FileMetaData));// read meta data
8012  is.skip(meta.nameSize); // skip grid name
8013  auto buffer = GridHandleT::BufferType::create(meta.gridSize, &pool);
8014  is.read((char*)buffer.data(), meta.gridSize);// read grid
8015  handles.emplace_back(std::move(buffer));
8016  }// loop over grids in segment
8017  }// loop over segments
8018  }
8019  return handles;
8020 } // readUncompressedGrids
8021 
8022 /// @brief Read a multiple un-compressed NanoVDB grids from a file and return them as a vector.
8023 template<typename GridHandleT, template<typename...> class VecT>
8024 VecT<GridHandleT> readUncompressedGrids(const char* fileName, const typename GridHandleT::BufferType& buffer = typename GridHandleT::BufferType())
8025 {
8026 #ifdef NANOVDB_USE_IOSTREAMS // use this to switch between std::ifstream or FILE implementations
8027  struct StreamT : public std::ifstream {
8028  StreamT(const char* name) : std::ifstream(name, std::ios::in | std::ios::binary){}
8029  void skip(int64_t off) { this->seekg(off, std::ios_base::cur); }
8030  };
8031 #else
8032  struct StreamT {
8033  FILE* fptr;
8034  StreamT(const char* name) { fptr = fopen(name, "rb"); }
8035  ~StreamT() { fclose(fptr); }
8036  bool read(char* data, size_t n) {
8037  size_t m = fread(data, 1, n, fptr);
8038  return n == m;
8039  }
8040  void skip(int64_t off) { fseek(fptr, (long int)off, SEEK_CUR); }
8041  bool is_open() const { return fptr != NULL; }
8042  };
8043 #endif
8044  StreamT is(fileName);
8045  if (!is.is_open()) {
8046  fprintf(stderr, "nanovdb::readUncompressedGrids: Unable to open file \"%s\"for input\n", fileName);
8047  exit(EXIT_FAILURE);
8048  }
8049  return readUncompressedGrids<GridHandleT, StreamT, VecT>(is, buffer);
8050 } // readUncompressedGrids
8051 
8052 #endif // if !defined(__CUDA_ARCH__) && !defined(__HIP__)
8053 
8054 } // namespace io
8055 
8056 // ----------------------------> Implementations of random access methods <--------------------------------------
8057 
8058 /// @brief Implements Tree::getValue(Coord), i.e. return the value associated with a specific coordinate @c ijk.
8059 /// @tparam BuildT Build type of the grid being called
8060 /// @details The value at a coordinate maps to the background, a tile value or a leaf value.
8061 template<typename BuildT>
8062 struct GetValue
8063 {
8064  __hostdev__ static auto get(const NanoRoot<BuildT>& root) { return root.mBackground; }
8065  __hostdev__ static auto get(const typename NanoRoot<BuildT>::Tile& tile) { return tile.value; }
8066  __hostdev__ static auto get(const NanoUpper<BuildT>& node, uint32_t n) { return node.mTable[n].value; }
8067  __hostdev__ static auto get(const NanoLower<BuildT>& node, uint32_t n) { return node.mTable[n].value; }
8068  __hostdev__ static auto get(const NanoLeaf<BuildT>& leaf, uint32_t n) { return leaf.getValue(n); } // works with all build types
8069 }; // GetValue<BuildT>
8070 
8071 template<typename BuildT>
8072 struct SetValue
8073 {
8074  static_assert(!BuildTraits<BuildT>::is_special, "SetValue does not support special value types");
8075  using ValueT = typename NanoLeaf<BuildT>::ValueType;
8076  __hostdev__ static auto set(NanoRoot<BuildT>&, const ValueT&) {} // no-op
8077  __hostdev__ static auto set(typename NanoRoot<BuildT>::Tile& tile, const ValueT& v) { tile.value = v; }
8078  __hostdev__ static auto set(NanoUpper<BuildT>& node, uint32_t n, const ValueT& v) { node.mTable[n].value = v; }
8079  __hostdev__ static auto set(NanoLower<BuildT>& node, uint32_t n, const ValueT& v) { node.mTable[n].value = v; }
8080  __hostdev__ static auto set(NanoLeaf<BuildT>& leaf, uint32_t n, const ValueT& v) { leaf.mValues[n] = v; }
8081 }; // SetValue<BuildT>
8082 
8083 template<typename BuildT>
8084 struct SetVoxel
8085 {
8086  static_assert(!BuildTraits<BuildT>::is_special, "SetVoxel does not support special value types");
8087  using ValueT = typename NanoLeaf<BuildT>::ValueType;
8088  __hostdev__ static auto set(NanoRoot<BuildT>&, const ValueT&) {} // no-op
8089  __hostdev__ static auto set(typename NanoRoot<BuildT>::Tile&, const ValueT&) {} // no-op
8090  __hostdev__ static auto set(NanoUpper<BuildT>&, uint32_t, const ValueT&) {} // no-op
8091  __hostdev__ static auto set(NanoLower<BuildT>&, uint32_t, const ValueT&) {} // no-op
8092  __hostdev__ static auto set(NanoLeaf<BuildT>& leaf, uint32_t n, const ValueT& v) { leaf.mValues[n] = v; }
8093 }; // SetVoxel<BuildT>
8094 
8095 /// @brief Implements Tree::isActive(Coord)
8096 /// @tparam BuildT Build type of the grid being called
8097 template<typename BuildT>
8098 struct GetState
8099 {
8100  __hostdev__ static auto get(const NanoRoot<BuildT>&) { return false; }
8101  __hostdev__ static auto get(const typename NanoRoot<BuildT>::Tile& tile) { return tile.state > 0; }
8102  __hostdev__ static auto get(const NanoUpper<BuildT>& node, uint32_t n) { return node.mValueMask.isOn(n); }
8103  __hostdev__ static auto get(const NanoLower<BuildT>& node, uint32_t n) { return node.mValueMask.isOn(n); }
8104  __hostdev__ static auto get(const NanoLeaf<BuildT>& leaf, uint32_t n) { return leaf.mValueMask.isOn(n); }
8105 }; // GetState<BuildT>
8106 
8107 /// @brief Implements Tree::getDim(Coord)
8108 /// @tparam BuildT Build type of the grid being called
8109 template<typename BuildT>
8110 struct GetDim
8111 {
8112  __hostdev__ static uint32_t get(const NanoRoot<BuildT>&) { return 0u; } // background
8113  __hostdev__ static uint32_t get(const typename NanoRoot<BuildT>::Tile&) { return 4096u; }
8114  __hostdev__ static uint32_t get(const NanoUpper<BuildT>&, uint32_t) { return 128u; }
8115  __hostdev__ static uint32_t get(const NanoLower<BuildT>&, uint32_t) { return 8u; }
8116  __hostdev__ static uint32_t get(const NanoLeaf<BuildT>&, uint32_t) { return 1u; }
8117 }; // GetDim<BuildT>
8118 
8119 /// @brief Return the pointer to the leaf node that contains Coord. Implements Tree::probeLeaf(Coord)
8120 /// @tparam BuildT Build type of the grid being called
8121 template<typename BuildT>
8122 struct GetLeaf
8123 {
8124  __hostdev__ static const NanoLeaf<BuildT>* get(const NanoRoot<BuildT>&) { return nullptr; }
8125  __hostdev__ static const NanoLeaf<BuildT>* get(const typename NanoRoot<BuildT>::Tile&) { return nullptr; }
8126  __hostdev__ static const NanoLeaf<BuildT>* get(const NanoUpper<BuildT>&, uint32_t) { return nullptr; }
8127  __hostdev__ static const NanoLeaf<BuildT>* get(const NanoLower<BuildT>&, uint32_t) { return nullptr; }
8128  __hostdev__ static const NanoLeaf<BuildT>* get(const NanoLeaf<BuildT>& leaf, uint32_t) { return &leaf; }
8129 }; // GetLeaf<BuildT>
8130 
8131 /// @brief Return point to the lower internal node where Coord maps to one of its values, i.e. terminates
8132 /// @tparam BuildT Build type of the grid being called
8133 template<typename BuildT>
8134 struct GetLower
8135 {
8136  __hostdev__ static const NanoLower<BuildT>* get(const NanoRoot<BuildT>&) { return nullptr; }
8137  __hostdev__ static const NanoLower<BuildT>* get(const typename NanoRoot<BuildT>::Tile&) { return nullptr; }
8138  __hostdev__ static const NanoLower<BuildT>* get(const NanoUpper<BuildT>&, uint32_t) { return nullptr; }
8139  __hostdev__ static const NanoLower<BuildT>* get(const NanoLower<BuildT>& node, uint32_t) { return &node; }
8140  __hostdev__ static const NanoLower<BuildT>* get(const NanoLeaf<BuildT>&, uint32_t) { return nullptr; }
8141 }; // GetLower<BuildT>
8142 
8143 /// @brief Return point to the upper internal node where Coord maps to one of its values, i.e. terminates
8144 /// @tparam BuildT Build type of the grid being called
8145 template<typename BuildT>
8146 struct GetUpper
8147 {
8148  __hostdev__ static const NanoUpper<BuildT>* get(const NanoRoot<BuildT>&) { return nullptr; }
8149  __hostdev__ static const NanoUpper<BuildT>* get(const typename NanoRoot<BuildT>::Tile&) { return nullptr; }
8150  __hostdev__ static const NanoUpper<BuildT>* get(const NanoUpper<BuildT>& node, uint32_t) { return &node; }
8151  __hostdev__ static const NanoUpper<BuildT>* get(const NanoLower<BuildT>& node, uint32_t) { return nullptr; }
8152  __hostdev__ static const NanoUpper<BuildT>* get(const NanoLeaf<BuildT>&, uint32_t) { return nullptr; }
8153 }; // GetUpper<BuildT>
8154 
8155 /// @brief Implements Tree::probeLeaf(Coord)
8156 /// @tparam BuildT Build type of the grid being called
8157 template<typename BuildT>
8158 struct ProbeValue
8159 {
8160  using ValueT = typename BuildToValueMap<BuildT>::Type;
8161  __hostdev__ static bool get(const NanoRoot<BuildT>& root, ValueT& v)
8162  {
8163  v = root.mBackground;
8164  return false;
8165  }
8166  __hostdev__ static bool get(const typename NanoRoot<BuildT>::Tile& tile, ValueT& v)
8167  {
8168  v = tile.value;
8169  return tile.state > 0u;
8170  }
8171  __hostdev__ static bool get(const NanoUpper<BuildT>& node, uint32_t n, ValueT& v)
8172  {
8173  v = node.mTable[n].value;
8174  return node.mValueMask.isOn(n);
8175  }
8176  __hostdev__ static bool get(const NanoLower<BuildT>& node, uint32_t n, ValueT& v)
8177  {
8178  v = node.mTable[n].value;
8179  return node.mValueMask.isOn(n);
8180  }
8181  __hostdev__ static bool get(const NanoLeaf<BuildT>& leaf, uint32_t n, ValueT& v)
8182  {
8183  v = leaf.getValue(n);
8184  return leaf.mValueMask.isOn(n);
8185  }
8186 }; // ProbeValue<BuildT>
8187 
8188 /// @brief Implements Tree::getNodeInfo(Coord)
8189 /// @tparam BuildT Build type of the grid being called
8190 template<typename BuildT>
8191 struct GetNodeInfo
8192 {
8195  struct NodeInfo
8196  {
8197  uint32_t level, dim;
8198  ValueType minimum, maximum;
8199  FloatType average, stdDevi;
8201  };
8202  __hostdev__ static NodeInfo get(const NanoRoot<BuildT>& root)
8203  {
8204  return NodeInfo{3u, NanoUpper<BuildT>::DIM, root.minimum(), root.maximum(), root.average(), root.stdDeviation(), root.bbox()};
8205  }
8206  __hostdev__ static NodeInfo get(const typename NanoRoot<BuildT>::Tile& tile)
8207  {
8208  return NodeInfo{3u, NanoUpper<BuildT>::DIM, tile.value, tile.value, static_cast<FloatType>(tile.value), 0, CoordBBox::createCube(tile.origin(), NanoUpper<BuildT>::DIM)};
8209  }
8210  __hostdev__ static NodeInfo get(const NanoUpper<BuildT>& node, uint32_t n)
8211  {
8212  return NodeInfo{2u, node.dim(), node.minimum(), node.maximum(), node.average(), node.stdDeviation(), node.bbox()};
8213  }
8214  __hostdev__ static NodeInfo get(const NanoLower<BuildT>& node, uint32_t n)
8215  {
8216  return NodeInfo{1u, node.dim(), node.minimum(), node.maximum(), node.average(), node.stdDeviation(), node.bbox()};
8217  }
8218  __hostdev__ static NodeInfo get(const NanoLeaf<BuildT>& leaf, uint32_t n)
8219  {
8220  return NodeInfo{0u, leaf.dim(), leaf.minimum(), leaf.maximum(), leaf.average(), leaf.stdDeviation(), leaf.bbox()};
8221  }
8222 }; // GetNodeInfo<BuildT>
8223 
8224 } // namespace nanovdb
8225 
8226 #endif // end of NANOVDB_NANOVDB_H_HAS_BEEN_INCLUDED
typename FloatTraits< BuildT >::FloatType FloatType
Definition: NanoVDB.h:5439
bool operator==(const Vec3 &rhs) const
Definition: NanoVDB.h:1561
void setAvg(const FloatType &)
Definition: NanoVDB.h:5820
ValueOffIterator beginValueOff() const
Definition: NanoVDB.h:6110
Map(double s, const Vec3d &t=Vec3d(0.0, 0.0, 0.0))
Definition: NanoVDB.h:3180
Vec3T applyInverseMapF(const Vec3T &xyz) const
Definition: NanoVDB.h:3601
typename BuildT::RootType RootType
Definition: NanoVDB.h:3702
int32_t ValueType
Definition: NanoVDB.h:1305
ValueT ValueType
Definition: NanoVDB.h:6708
static size_t memUsage(uint32_t bitWidth)
Definition: NanoVDB.h:5688
bool type
Definition: NanoVDB.h:689
Visits all tile values in this node, i.e. both inactive and active tiles.
Definition: NanoVDB.h:5038
T lengthSqr() const
Definition: NanoVDB.h:1772
const ValueT & getMin() const
Definition: NanoVDB.h:4308
void setOrigin(const T &ijk)
Definition: NanoVDB.h:5865
CoordType getCoord() const
Definition: NanoVDB.h:4376
uint16_t ArrayType
Definition: NanoVDB.h:5964
bool operator==(const Coord &rhs) const
Definition: NanoVDB.h:1394
static uint32_t CoordToOffset(const CoordT &ijk)
Return the linear offset corresponding to the given coordinate.
Definition: NanoVDB.h:6297
FloatType mStdDevi
Definition: NanoVDB.h:5451
bool empty() const
Definition: NanoVDB.h:2333
float type
Definition: NanoVDB.h:696
Trait use to remove reference, i.e. "&", qualifier from a type. Default implementation is just a pass...
Definition: NanoVDB.h:581
void writeUncompressedGrids(const char *fileName, const VecT< GridHandleT > &handles, bool raw=false)
write multiple NanoVDB grids to a single file, without compression.
Definition: NanoVDB.h:7951
const RootT & root() const
Definition: NanoVDB.h:7221
bool isEmpty() const
test if the grid is empty, e.i the root table has size 0
Definition: NanoVDB.h:3679
ValueType minimum() const
Return a const reference to the minimum active value encoded in this leaf node.
Definition: NanoVDB.h:6182
typename RootType::LeafNodeType LeafNodeType
Definition: NanoVDB.h:4002
ValueType getValue(const CoordType &ijk) const
Return the value of the given voxel.
Definition: NanoVDB.h:4632
Definition: NanoVDB.h:7873
ValueType maximum() const
Return a const reference to the maximum active value encoded in this leaf node.
Definition: NanoVDB.h:6185
bool isLevelSet() const
Definition: NanoVDB.h:7556
StatsT mStdDevi
Definition: NanoVDB.h:4886
const RootT & root() const
Definition: NanoVDB.h:4033
const Vec3T & min() const
Definition: NanoVDB.h:2241
static bool hasStats()
Definition: NanoVDB.h:5463
Vec3(T x)
Definition: NanoVDB.h:1538
ValueType getValue(const CoordT &ijk) const
Return the voxel value at the given coordinate.
Definition: NanoVDB.h:6254
static BBox createCube(const CoordT &min, typename CoordT::ValueType dim)
Definition: NanoVDB.h:2441
static bool isAligned(const void *p)
return true if the specified pointer is aligned
Definition: NanoVDB.h:737
GridClass mapToGridClass(GridClass defaultClass=GridClass::Unknown)
Maps from a templated build type to a GridClass enum.
Definition: NanoVDB.h:2110
Definition: IndexIterator.h:42
BBox(BBox &other, const SplitT &)
Definition: NanoVDB.h:2432
static constexpr uint32_t padding()
Return padding of this class in bytes, due to aliasing and 32B alignment.
Definition: NanoVDB.h:5527
bool operator!=(const Iterator &rhs) const
Definition: NanoVDB.h:2401
FloatType variance() const
Return the variance of all the active values encoded in this leaf node.
Definition: NanoVDB.h:6191
static float value()
Definition: NanoVDB.h:1039
ChildIter(RootT *parent)
Definition: NanoVDB.h:4395
uint32_t operator*() const
Definition: NanoVDB.h:2900
Vec3T applyJacobian(const Vec3T &xyz) const
Definition: NanoVDB.h:3592
void setBitOn(std::initializer_list< uint8_t > list)
Definition: NanoVDB.h:2752
uint32_t countOn(uint32_t i) const
Return the number of lower set bits in mask up to but excluding the i&#39;th bit.
Definition: NanoVDB.h:2849
GridBlindDataClass mDataClass
Definition: NanoVDB.h:3337
int age() const
Check the major version of this instance relative to NANOVDB_MAJOR_VERSION_NUMBER.
Definition: NanoVDB.h:978
Tile * tile(uint32_t n)
Definition: NanoVDB.h:4258
ValueType operator*() const
Definition: NanoVDB.h:6065
void setDev(float dev)
Definition: NanoVDB.h:5562
uint64_t lastOffset() const
Definition: NanoVDB.h:5880
Vec3T indexToWorldF(const Vec3T &xyz) const
index to world space transformation
Definition: NanoVDB.h:3796
const StatsT & average() const
Definition: NanoVDB.h:4944
Coord & operator+=(const Coord &rhs)
Definition: NanoVDB.h:1427
bool probeValue(const CoordType &ijk, ValueType &v) const
return the state and updates the value of the specified voxel
Definition: NanoVDB.h:5219
VDB Tree, which is a thin wrapper around a RootNode.
Definition: NanoVDB.h:3989
decltype(mFlags) Type
Definition: NanoVDB.h:2714
void setOrigin(const T &ijk)
Definition: NanoVDB.h:6010
uint8_t * getRoot()
Definition: NanoVDB.h:3954
const BlindDataT * getBlindData() const
Get a const pointer to the blind data represented by this meta data.
Definition: NanoVDB.h:3358
const MaskType< LOG2DIM > & valueMask() const
Return a const reference to the bit mask of active voxels in this internal node.
Definition: NanoVDB.h:5172
ChildIter operator++(int)
Definition: NanoVDB.h:4420
static BBox createCube(typename CoordT::ValueType min, typename CoordT::ValueType max)
Definition: NanoVDB.h:2446
FloatType getDev() const
Definition: NanoVDB.h:5477
bool getMin() const
Definition: NanoVDB.h:5759
Vec3(const Vec3T< T2 > &v)
Definition: NanoVDB.h:1547
ValueT getValue(uint32_t n) const
Definition: NanoVDB.h:4925
void setOn()
Set all bits on.
Definition: NanoVDB.h:3039
CoordT getCoord() const
Definition: NanoVDB.h:6137
float Type
Definition: NanoVDB.h:716
static uint64_t alignmentPadding(const void *p)
return the smallest number of bytes that when added to the specified pointer results in an aligned po...
Definition: NanoVDB.h:749
float FloatType
Definition: NanoVDB.h:5509
Highest level of the data structure. Contains a tree and a world->index transform (that currently onl...
Definition: NanoVDB.h:3698
T lengthSqr() const
Definition: NanoVDB.h:1583
Vec3 & normalize()
Definition: NanoVDB.h:1633
auto getNodeInfo(const CoordType &ijk) const
Definition: NanoVDB.h:7293
DataType * data()
Definition: NanoVDB.h:4589
Vec4 & normalize()
Definition: NanoVDB.h:1809
Vec4(T x, T y, T z, T w)
Definition: NanoVDB.h:1740
Vec4 & operator-=(const Vec4 &v)
Definition: NanoVDB.h:1792
DenseIterator beginDense() const
Definition: NanoVDB.h:5152
Vec3T applyMapF(const Vec3T &ijk) const
Apply the forward affine transformation to a vector using 32bit floating point arithmetics.
Definition: NanoVDB.h:3220
uint32_t findFirst() const
Definition: NanoVDB.h:3107
ValueType operator()(const CoordType &ijk) const
Definition: NanoVDB.h:6609
void setOn(uint32_t offset)
Definition: NanoVDB.h:5817
FloatType stdDevi
Definition: NanoVDB.h:8199
typename Vec3T::ValueType ValueType
Definition: NanoVDB.h:2305
void setMaskOn(MaskT mask)
Definition: NanoVDB.h:2764
bool isOff() const
Return true if none of the bits are set in this Mask.
Definition: NanoVDB.h:2998
const ValueType & operator[](IndexType i) const
Return a const reference to the given Coord component.
Definition: NanoVDB.h:1347
void setDev(const StatsT &v)
Definition: NanoVDB.h:4316
const BBox< Vec3d > & worldBBox() const
return AABB of active values in world space
Definition: NanoVDB.h:3664
#define NANOVDB_PATCH_VERSION_NUMBER
Definition: NanoVDB.h:135
const RootT & root() const
Definition: NanoVDB.h:6742
bool isBreadthFirst() const
Definition: NanoVDB.h:3842
bool isSequential() const
return true if the specified node type is layed out breadth-first in memory and has a fixed size...
Definition: NanoVDB.h:3847
static constexpr uint64_t memUsage()
Definition: NanoVDB.h:5583
uint64_t activeVoxelCount() const
Return a const reference to the index bounding box of all the active values in this tree...
Definition: NanoVDB.h:4065
T Sign(const T &x)
Return the sign of the given value as an integer (either -1, 0 or 1).
Definition: NanoVDB.h:1245
DenseIterator(const InternalNode *parent)
Definition: NanoVDB.h:5122
const ValueType & background() const
Return the total number of active voxels in the root and all its child nodes.
Definition: NanoVDB.h:4600
Rgba8(uint8_t v)
ctor where all channels are initialized to the same value
Definition: NanoVDB.h:1901
Metafunction used to determine if the first template parameter is a specialization of the class templ...
Definition: NanoVDB.h:637
bool probeValue(const CoordType &ijk, ValueType &v) const
Definition: NanoVDB.h:6766
uint32_t operator*() const
Definition: NanoVDB.h:2872
Trait used to transfer the const-ness of a reference type to another type.
Definition: NanoVDB.h:611
static uint32_t dim()
Definition: NanoVDB.h:6033
bool getDev() const
Definition: NanoVDB.h:5815
Class to access values in channels at a specific voxel location.
Definition: NanoVDB.h:7723
Vec3T worldToIndexDir(const Vec3T &dir) const
transformation from world space direction to index space direction
Definition: NanoVDB.h:3783
static float value()
Definition: NanoVDB.h:1023
const ValueT & getMax() const
Definition: NanoVDB.h:4943
Mask(bool on)
Definition: NanoVDB.h:2934
Vec4(const Vec4< T2 > &v)
Definition: NanoVDB.h:1745
bool isFogVolume() const
Definition: NanoVDB.h:3830
typename GridT::TreeType Type
Definition: NanoVDB.h:3975
const ValueType & maximum() const
Return a const reference to the maximum active value encoded in this internal node and any of its chi...
Definition: NanoVDB.h:5186
char mGridName[MaxNameSize]
Definition: NanoVDB.h:3521
Coord operator+(const Coord &rhs) const
Definition: NanoVDB.h:1424
const uint8_t * getRoot() const
Definition: NanoVDB.h:3955
Type getFlags() const
Definition: NanoVDB.h:2744
static uint32_t CoordToOffset(const CoordType &ijk)
Return the linear offset corresponding to the given coordinate.
Definition: NanoVDB.h:5262
Version version() const
Definition: NanoVDB.h:3720
bool isFloatingPointVector(GridType gridType)
return true if the GridType maps to a floating point vec3.
Definition: NanoVDB.h:831
PointAccessor(const NanoGrid< Point > &grid)
Definition: NanoVDB.h:7662
Vec3T indexToWorldGradF(const Vec3T &grad) const
Transforms the gradient from index space to world space.
Definition: NanoVDB.h:3811
typename ChildT::CoordType CoordType
Definition: NanoVDB.h:4977
bool isEmpty() const
Definition: NanoVDB.h:7584
MaskT< LOG2DIM > mMask
Definition: NanoVDB.h:5937
GridClass
Classes (superset of OpenVDB) that are currently supported by NanoVDB.
Definition: NanoVDB.h:362
typename DataType::ValueT ValueType
Definition: NanoVDB.h:4339
static bool safeCast(const GridData *gridData)
return true if it is safe to cast the grid to a pointer of type GridMetaData, i.e. construction can be avoided.
Definition: NanoVDB.h:7545
uint64_t magic
Definition: NanoVDB.h:7849
bool updateBBox()
Updates the local bounding box of active voxels in this node. Return true if bbox was updated...
Definition: NanoVDB.h:6400
Vec3 operator*(const Vec3 &v) const
Definition: NanoVDB.h:1589
static T scalar(const T &s)
Definition: NanoVDB.h:1976
Vec3T indexToWorldDirF(const Vec3T &dir) const
transformation from index space direction to world space direction
Definition: NanoVDB.h:3801
Vec3T applyJacobianF(const Vec3T &ijk) const
Apply the linear forward 3x3 transformation to an input 3d vector using 32bit floating point arithmet...
Definition: NanoVDB.h:3238
typename RootT::BuildType BuildType
Definition: NanoVDB.h:4004
void setBitOff(uint8_t bit)
Definition: NanoVDB.h:2750
const uint32_t & packed() const
Definition: NanoVDB.h:1943
void setMask(uint32_t offset, bool v)
Definition: NanoVDB.h:5940
Definition: NanoVDB.h:4435
ValueType min() const
Return the smallest vector component.
Definition: NanoVDB.h:1658
ValueType operator*() const
Definition: NanoVDB.h:4500
uint32_t state
Definition: NanoVDB.h:4246
void setAvg(const FloatType &)
Definition: NanoVDB.h:6006
BuildT BuildType
Definition: NanoVDB.h:5438
bool isActive(const CoordType &ijk) const
Return the active state of the given voxel (regardless of state or location in the tree...
Definition: NanoVDB.h:4047
typename UpperNodeType::ChildNodeType LowerNodeType
Definition: NanoVDB.h:4001
Return the pointer to the leaf node that contains Coord. Implements Tree::probeLeaf(Coord) ...
Definition: NanoVDB.h:3479
Vec3 operator+(const Vec3 &v) const
Definition: NanoVDB.h:1591
const MaskType< LOG2DIM > & getChildMask() const
Definition: NanoVDB.h:5177
bool isInside(const Vec3T &p) const
Definition: NanoVDB.h:2340
const ValueType & minimum() const
Return a const reference to the minimum active value encoded in this internal node and any of its chi...
Definition: NanoVDB.h:5183
typename match_const< DataType, RootT >::type DataT
Definition: NanoVDB.h:4355
const GridBlindMetaData * blindMetaData(uint32_t n) const
Returns a const reference to the blindMetaData at the specified linear offset.
Definition: NanoVDB.h:3638
bool hasBBox() const
Definition: NanoVDB.h:6284
uint8_t mFlags
Definition: NanoVDB.h:5445
ChildIterator beginChild()
Definition: NanoVDB.h:5034
uint64_t mOffset
Definition: NanoVDB.h:5972
static uint64_t memUsage(uint32_t tableSize)
Return the expected memory footprint in bytes with the specified number of tiles. ...
Definition: NanoVDB.h:4622
Vec3T applyInverseJacobian(const Vec3T &xyz) const
Definition: NanoVDB.h:3594
void setBlindData(void *blindData)
Definition: NanoVDB.h:3348
static constexpr uint32_t padding()
Definition: NanoVDB.h:5851
Trait use to remove pointer, i.e. "*", qualifier from a type. Default implementation is just a pass-t...
Definition: NanoVDB.h:595
ValueType getFirstValue() const
Return the first value in this leaf node.
Definition: NanoVDB.h:6257
Definition: NanoVDB.h:3688
LeafNodeType Node0
Definition: NanoVDB.h:4011
static constexpr uint32_t padding()
Definition: NanoVDB.h:5584
ValueType operator()(const CoordType &ijk) const
Definition: NanoVDB.h:7291
uint64_t idx(int i, int j, int k) const
Definition: NanoVDB.h:7788
bool hasStats() const
Definition: NanoVDB.h:5856
void setMin(const ValueT &v)
Definition: NanoVDB.h:4313
Return point to the upper internal node where Coord maps to one of its values, i.e. terminates.
Definition: NanoVDB.h:8146
typename DataType::StatsT FloatType
Definition: NanoVDB.h:4340
Vec3 cross(const Vec3T &v) const
Definition: NanoVDB.h:1577
bool isLevelSet() const
Definition: NanoVDB.h:3829
Implements Tree::getValue(Coord), i.e. return the value associated with a specific coordinate ijk...
Definition: NanoVDB.h:3469
ValueType & operator[](IndexType i)
Return a non-const reference to the given Coord component.
Definition: NanoVDB.h:1351
const DataType * data() const
Definition: NanoVDB.h:6175
ValueType operator*() const
Definition: NanoVDB.h:5055
BitFlags()
Definition: NanoVDB.h:2715
ChildT ChildNodeType
Definition: NanoVDB.h:4333
static constexpr uint32_t padding()
Definition: NanoVDB.h:5806
#define NANOVDB_MAGIC_GRID
Definition: NanoVDB.h:127
void set(const MatT &mat, const MatT &invMat, const Vec3T &translate, double taper=1.0)
Initialize the member data from 3x3 or 4x4 matrices.
Definition: NanoVDB.h:3296
Vec4 operator*(const Vec4 &v) const
Definition: NanoVDB.h:1778
const LeafT * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:6993
ChildIter(ParentT *parent)
Definition: NanoVDB.h:5007
bool is_divisible() const
Definition: NanoVDB.h:2451
BaseBBox()
Definition: NanoVDB.h:2287
uint64_t memUsage() const
Return the actual memory footprint of this root node.
Definition: NanoVDB.h:4625
Vec4 & operator/=(const T &s)
Definition: NanoVDB.h:1808
MaskT< LOG2DIM > ArrayType
Definition: NanoVDB.h:5745
BitFlags(std::initializer_list< uint8_t > list)
Definition: NanoVDB.h:2716
T Type
Definition: NanoVDB.h:653
void setMin(const ValueType &)
Definition: NanoVDB.h:6004
uint64_t mMagic
Definition: NanoVDB.h:3514
void setBBoxOn(bool on=true)
Definition: NanoVDB.h:3573
uint64_t mGridSize
Definition: NanoVDB.h:3520
const char * shortGridName() const
Definition: NanoVDB.h:7573
void setOrigin(const T &ijk)
Definition: NanoVDB.h:4940
Vec3 operator+(const Coord &ijk) const
Definition: NanoVDB.h:1593
const LeafT * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:7296
NodeT * probeChild(ValueType &value) const
Definition: NanoVDB.h:4543
Vec3 operator/(const Vec3 &v) const
Definition: NanoVDB.h:1590
RootT Node3
Definition: NanoVDB.h:4008
bool isCompatible() const
Definition: NanoVDB.h:974
Trait to map from LEVEL to node type.
Definition: NanoVDB.h:6453
bool hasMinMax() const
Definition: NanoVDB.h:7564
bool isValid() const
Definition: NanoVDB.h:7553
CoordType getOrigin() const
Definition: NanoVDB.h:5060
Vec3T & min()
Definition: NanoVDB.h:2239
ValueType operator()(int i, int j, int k) const
Definition: NanoVDB.h:7292
Rgba8(float r, float g, float b, float a=1.0f)
floating-point r,g,b,a ctor where alpha channel defaults to opaque
Definition: NanoVDB.h:1908
void extrema(ValueType &min, ValueType &max) const
Sets the extrema values of all the active values in this tree, i.e. in all nodes of the tree...
Definition: NanoVDB.h:4162
const LeafNodeType * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:4637
FloatType variance() const
Return the variance of all the active values encoded in this root node and any of its child nodes...
Definition: NanoVDB.h:4616
bool probeValue(const CoordType &ijk, ValueType &v) const
Return true if this tree is empty, i.e. contains no values or nodes.
Definition: NanoVDB.h:4053
#define NANOVDB_MINOR_VERSION_NUMBER
Definition: NanoVDB.h:134
const void * blindData() const
Definition: NanoVDB.h:3351
bool operator<(const Coord &rhs) const
Return true if this Coord is lexicographically less than the given Coord.
Definition: NanoVDB.h:1374
CoordType getOrigin() const
Definition: NanoVDB.h:5099
A simple vector class with four components, similar to openvdb::math::Vec4.
Definition: NanoVDB.h:1727
CoordT RoundDown(const Vec3T< RealT > &xyz)
Definition: NanoVDB.h:1226
ValueIter & operator++()
Definition: NanoVDB.h:4461
uint64_t KeyT
Return a key based on the coordinates of a voxel.
Definition: NanoVDB.h:4184
Vec3d mVoxelSize
Definition: NanoVDB.h:3524
void setValueOnly(const CoordT &ijk, const ValueType &v)
Definition: NanoVDB.h:6270
void setOn(uint32_t n)
Set the specified bit on.
Definition: NanoVDB.h:3007
const NodeT * getFirstNode() const
return a const pointer to the first node of the specified type
Definition: NanoVDB.h:4110
const uint8_t & r() const
Definition: NanoVDB.h:1945
bool isActive(uint32_t n) const
Definition: NanoVDB.h:4931
BuildT ValueType
Definition: NanoVDB.h:5437
uint64_t mFlags
Definition: NanoVDB.h:4879
int64_t mDataOffset
Definition: NanoVDB.h:3333
bool isMask() const
Definition: NanoVDB.h:3835
int findBlindData(const char *name) const
Return the index of the first blind data with specified name if found, otherwise -1.
Definition: NanoVDB.h:3920
const LeafNodeType * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:5220
ReadAccessor(const GridT &grid)
Constructor from a grid.
Definition: NanoVDB.h:7210
MatType scale(const Vec3< typename MatType::value_type > &s)
Return a matrix that scales by s.
Definition: Mat.h:615
auto getNodeInfo(const CoordType &ijk) const
Definition: NanoVDB.h:6990
T type
Definition: NanoVDB.h:563
static constexpr uint32_t padding()
Return padding of this class in bytes, due to aliasing and 32B alignment.
Definition: NanoVDB.h:4218
Implements Tree::getNodeInfo(Coord)
Definition: NanoVDB.h:3483
static uint32_t FindHighestOn(uint32_t v)
Returns the index of the highest, i.e. most significant, on bit in the specified 32 bit word...
Definition: NanoVDB.h:2576
uint64_t voxelCount
Definition: NanoVDB.h:7875
Rgba8()
Default ctor initializes all channels to zero.
Definition: NanoVDB.h:1886
const CoordT & operator*() const
Definition: NanoVDB.h:2418
Tile * probeTile(const CoordT &ijk)
Definition: NanoVDB.h:4264
static fileSize_t write(std::ostream &os, const GridHandle< BufferT > &handle, Codec codec, uint32_t n)
bool operator>=(const Version &rhs) const
Definition: NanoVDB.h:969
uint64_t getAvg() const
Definition: NanoVDB.h:5884
int32_t z() const
Definition: NanoVDB.h:1333
CoordT getCoord() const
Definition: NanoVDB.h:6070
ValueType minimum
Definition: NanoVDB.h:8198
ChildT UpperNodeType
Definition: NanoVDB.h:4336
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:6991
uint32_t mGridCount
Definition: NanoVDB.h:3519
uint64_t getAvg() const
Definition: NanoVDB.h:5910
CoordT mBBoxMin
Definition: NanoVDB.h:5443
const char * gridName() const
Definition: NanoVDB.h:3644
BBox(const Coord &min, const Coord &max)
Definition: NanoVDB.h:2319
Vec3T applyIJTF(const Vec3T &xyz) const
Definition: NanoVDB.h:3289
uint64_t FloatType
Definition: NanoVDB.h:2019
uint8_t & r()
Definition: NanoVDB.h:1949
const MaskType< LOG2DIM > & childMask() const
Return a const reference to the bit mask of child nodes in this internal node.
Definition: NanoVDB.h:5176
void setAverageOn(bool on=true)
Definition: NanoVDB.h:3575
static constexpr uint8_t bitWidth()
Definition: NanoVDB.h:5590
const GridType & gridType() const
Definition: NanoVDB.h:7554
ValueIterator beginValue()
Definition: NanoVDB.h:4480
BBox expandBy(typename CoordT::ValueType padding) const
Return a new instance that is expanded by the specified padding.
Definition: NanoVDB.h:2490
ValueType getValue(const CoordType &ijk) const
Return the value of the given voxel (regardless of state or location in the tree.) ...
Definition: NanoVDB.h:4043
const TreeT & tree() const
Return a const reference to the tree.
Definition: NanoVDB.h:3753
ValueType operator*() const
Definition: NanoVDB.h:6098
Definition: NanoVDB.h:8195
typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType type
Definition: NanoVDB.h:3427
uint64_t FloatType
Definition: NanoVDB.h:2013
ValueIter(RootT *parent)
Definition: NanoVDB.h:4444
NodeT * getFirstNode()
return a pointer to the first node of the specified type
Definition: NanoVDB.h:4100
float mQuantum
Definition: NanoVDB.h:5517
double FloatType
Definition: NanoVDB.h:2043
const T & operator[](int i) const
Definition: NanoVDB.h:1768
Iterator & operator++()
Definition: NanoVDB.h:2875
static const int MaxNameSize
Definition: NanoVDB.h:3332
bool isPointIndex() const
Definition: NanoVDB.h:3832
BBox(const Vec3T &min, const Vec3T &max)
Definition: NanoVDB.h:2315
uint64_t gridPoints(const AttT *&begin, const AttT *&end) const
Return the total number of point in the grid and set the iterators to the complete range of points...
Definition: NanoVDB.h:7683
Vec3(const Coord &ijk)
Definition: NanoVDB.h:1557
Map mMap
Definition: NanoVDB.h:3522
void setValue(uint32_t n, const ValueT &v)
Definition: NanoVDB.h:4907
#define NANOVDB_MAGIC_FILE
Definition: NanoVDB.h:128
ValueIterator(const InternalNode *parent)
Definition: NanoVDB.h:5049
float type
Definition: NanoVDB.h:710
TreeT & tree()
Return a non-const reference to the tree.
Definition: NanoVDB.h:3756
GridType mapToGridType()
Maps from a templated build type to a GridType enum.
Definition: NanoVDB.h:2050
static uint64_t memUsage()
Definition: NanoVDB.h:5461
#define __device__
Definition: NanoVDB.h:219
typename ChildT::LeafNodeType LeafNodeType
Definition: NanoVDB.h:4338
Bit-compacted representation of all three version numbers.
Definition: NanoVDB.h:947
int32_t x() const
Definition: NanoVDB.h:1331
ValueType getMax() const
Definition: NanoVDB.h:6000
ReadAccessor(const TreeT &tree)
Constructor from a tree.
Definition: NanoVDB.h:6730
BBox< Vec3< RealT > > asReal() const
Definition: NanoVDB.h:2483
DataType * data()
Definition: NanoVDB.h:4019
bool isOn() const
Return true if all the bits are set in this Mask.
Definition: NanoVDB.h:2989
bool isValueOn() const
Definition: NanoVDB.h:4555
bool hasStdDeviation() const
Definition: NanoVDB.h:7568
Type Max(Type a, Type b)
Definition: NanoVDB.h:1110
typename DataType::ValueT ValueType
Definition: NanoVDB.h:4972
const StatsT & stdDeviation() const
Definition: NanoVDB.h:4945
typename GridOrTreeOrRootT::LeafNodeType Type
Definition: NanoVDB.h:3411
Coord operator>>(IndexType n) const
Definition: NanoVDB.h:1371
T ValueType
Definition: NanoVDB.h:1734
static constexpr uint32_t padding()
Definition: NanoVDB.h:5620
Definition: NanoVDB.h:4867
const Vec3d & voxelSize() const
Return a vector of the axial voxel sizes.
Definition: NanoVDB.h:7768
const MaskType< LOG2DIM > & getValueMask() const
Definition: NanoVDB.h:6179
void setAvg(const bool &)
Definition: NanoVDB.h:5771
Vec3(const Vec3< T2 > &v)
Definition: NanoVDB.h:1553
bool isStaggered() const
Definition: NanoVDB.h:3831
static uint64_t memUsage()
return memory usage in bytes for the class
Definition: NanoVDB.h:4024
CoordT offsetToGlobalCoord(uint32_t n) const
Definition: NanoVDB.h:6214
uint64_t AlignUp(uint64_t byteCount)
round up byteSize to the nearest wordSize, e.g. to align to machine word: AlignUp<sizeof(size_t)(n) ...
Definition: NanoVDB.h:1288
void setMin(const ValueType &)
Definition: NanoVDB.h:5859
Type Min(Type a, Type b)
Definition: NanoVDB.h:1089
static constexpr uint32_t SIZE
Definition: NanoVDB.h:2827
uint32_t mNodeCount[3]
Definition: NanoVDB.h:3943
BaseBBox & intersect(const BaseBBox &bbox)
Intersect this bounding box with the given bounding box.
Definition: NanoVDB.h:2266
ValueIterator cbeginValueAll() const
Definition: NanoVDB.h:6162
void setMin(float min)
Definition: NanoVDB.h:5553
ValueType mMaximum
Definition: NanoVDB.h:5449
Coord round() const
Round each component if this Vec<T> to its closest integer value.
Definition: NanoVDB.h:1675
BaseBBox(const Vec3T &min, const Vec3T &max)
Definition: NanoVDB.h:2288
typename GridOrTreeOrRootT::LeafNodeType type
Definition: NanoVDB.h:3412
uint64_t * words()
Return a pointer to the list of words of the bit mask.
Definition: NanoVDB.h:2949
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5801
Trait use to const from type. Default implementation is just a pass-through.
Definition: NanoVDB.h:561
Coord operator&(IndexType n) const
Return a new instance with coordinates masked by the given unsigned integer.
Definition: NanoVDB.h:1365
void init(float min, float max, uint8_t bitWidth)
Definition: NanoVDB.h:5531
const T & operator[](int i) const
Definition: NanoVDB.h:1572
ValueType getValue(const CoordType &ijk) const
Return the value of the given voxel.
Definition: NanoVDB.h:5216
bool isActive(const CoordT &ijk) const
Return true if the voxel value at the given coordinate is active.
Definition: NanoVDB.h:6273
const NodeT * getNode() const
Return a const point to the cached node of the specified type.
Definition: NanoVDB.h:7232
typename DataType::BuildT BuildType
Definition: NanoVDB.h:4341
bool isPointData() const
Definition: NanoVDB.h:3834
void setMin(const bool &)
Definition: NanoVDB.h:5769
uint32_t getDim(const CoordType &ijk, const RayT &ray) const
Definition: NanoVDB.h:7433
uint64_t activeVoxelCount() const
Computes a AABB of active values in world space.
Definition: NanoVDB.h:3823
void setValue(uint32_t offset, const ValueType &value)
Definition: NanoVDB.h:5467
Vec3T worldToIndexF(const Vec3T &xyz) const
world to index space transformation
Definition: NanoVDB.h:3792
bool operator!=(const Mask &other) const
Definition: NanoVDB.h:2980
Vec3T matMult(const float *mat, const Vec3T &xyz)
Multiply a 3x3 matrix and a 3d vector using 32bit floating point arithmetics.
Definition: NanoVDB.h:2133
int32_t Ceil(float x)
Definition: NanoVDB.h:1158
ConstDenseIterator cbeginChildAll() const
Definition: NanoVDB.h:4579
void initBit(std::initializer_list< uint8_t > list)
Definition: NanoVDB.h:2729
uint64_t getMin() const
Definition: NanoVDB.h:5882
bool hasAverage() const
Definition: NanoVDB.h:7567
void setMin(const ValueType &v)
Definition: NanoVDB.h:5479
ReadAccessor(const TreeT &tree)
Constructor from a tree.
Definition: NanoVDB.h:7216
bool FloatType
Definition: NanoVDB.h:2007
bool isMaskOff(std::initializer_list< MaskT > list) const
return true if any of the masks in the list are off
Definition: NanoVDB.h:2804
uint32_t gridIndex() const
Definition: NanoVDB.h:7571
void setStdDeviationOn(bool on=true)
Definition: NanoVDB.h:3576
Definition: NanoVDB.h:1993
typename DataType::FloatType FloatType
Definition: NanoVDB.h:6038
#define NANOVDB_DATA_ALIGNMENT
Definition: NanoVDB.h:154
Vec3T applyInverseMap(const Vec3T &xyz) const
Definition: NanoVDB.h:3590
typename DataType::Tile Tile
Definition: NanoVDB.h:4346
Map()
Default constructor for the identity map.
Definition: NanoVDB.h:3169
static uint64_t memUsage()
Return memory usage in bytes for this class only.
Definition: NanoVDB.h:3661
const RootT & root() const
Definition: NanoVDB.h:6935
Definition: NanoVDB.h:2708
const T * asPointer() const
return a const raw constant pointer to array of three vector components
Definition: NanoVDB.h:1689
ValueType operator()(int i, int j, int k) const
Definition: NanoVDB.h:6610
Trait used to identify template parameter that are pointers.
Definition: NanoVDB.h:534
DataType * data()
Definition: NanoVDB.h:5161
bool isBitOff(uint8_t bit) const
Definition: NanoVDB.h:2788
bool getValue(uint32_t i) const
Definition: NanoVDB.h:5811
bool isChild(uint32_t n) const
Definition: NanoVDB.h:4937
bool isInside(const Vec3T &xyz)
Definition: NanoVDB.h:2277
Dummy type for a 16bit quantization of float point values.
Definition: NanoVDB.h:292
CoordType getCoord() const
Definition: NanoVDB.h:5065
uint8_t ArrayType
Definition: NanoVDB.h:5616
float asFloat(int n) const
return n&#39;th color channel as a float in the range 0 to 1
Definition: NanoVDB.h:1940
bool isSequential() const
return true if nodes at all levels can safely be accessed with simple linear offsets ...
Definition: NanoVDB.h:3855
typename Mask< Log2Dim >::template Iterator< On > MaskIterT
Definition: NanoVDB.h:4982
const FloatType & average() const
Return a const reference to the average of all the active values encoded in this root node and any of...
Definition: NanoVDB.h:4613
ValueOnIterator()
Definition: NanoVDB.h:6054
CoordT mBBoxMin
Definition: NanoVDB.h:5967
const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType Type
Definition: NanoVDB.h:3447
Coord & operator-=(const Coord &rhs)
Definition: NanoVDB.h:1434
bool isActive() const
Return true if any of the voxel value are active in this leaf node.
Definition: NanoVDB.h:6277
BBox< CoordT > bbox() const
Return the bounding box in index space of active values in this leaf node.
Definition: NanoVDB.h:6223
uint32_t rootTableSize() const
Definition: NanoVDB.h:7583
static constexpr uint8_t bitWidth()
Definition: NanoVDB.h:5626
NodeT & operator*() const
Definition: NanoVDB.h:5013
ValueIter operator++(int)
Definition: NanoVDB.h:4469
uint64_t memUsage() const
return memory usage in bytes for the leaf node
Definition: NanoVDB.h:6242
ConstDenseIterator cbeginDense() const
Definition: NanoVDB.h:4578
CoordT origin() const
Definition: NanoVDB.h:4243
FloatType getDev() const
Definition: NanoVDB.h:6002
const BBox< Vec3d > & worldBBox() const
Definition: NanoVDB.h:7575
ReadAccessor(const RootT &root)
Constructor from a root node.
Definition: NanoVDB.h:6899
const StatsT & stdDeviation() const
Definition: NanoVDB.h:4311
T type
Definition: NanoVDB.h:507
bool isMask() const
Definition: NanoVDB.h:7562
Vec3 operator-(const Coord &ijk) const
Definition: NanoVDB.h:1594
Vec3< T2 > operator/(T1 scalar, const Vec3< T2 > &vec)
Definition: NanoVDB.h:1698
static uint64_t memUsage()
Definition: NanoVDB.h:5949
BBox< Vec3d > mWorldBBox
Definition: NanoVDB.h:3523
uint64_t ValueType
Definition: NanoVDB.h:5841
uint16_t ArrayType
Definition: NanoVDB.h:5646
uint32_t nodeCount() const
Definition: NanoVDB.h:4079
const uint64_t & valueCount() const
Return total number of values indexed by the IndexGrid.
Definition: NanoVDB.h:7771
ValueT value
Definition: NanoVDB.h:4247
Node caching at all (three) tree levels.
Definition: NanoVDB.h:7165
BBox()
Default construction sets BBox to an empty bbox.
Definition: NanoVDB.h:2310
Definition: NanoVDB.h:3471
const DataType * data() const
Definition: NanoVDB.h:3724
bool isActive() const
Definition: NanoVDB.h:5066
typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType Type
Definition: NanoVDB.h:3426
void setValueOnly(uint32_t offset, uint16_t value)
Definition: NanoVDB.h:5991
BuildT BuildType
Definition: NanoVDB.h:7189
Stuct with all the member data of the LeafNode (useful during serialization of an openvdb LeafNode) ...
Definition: NanoVDB.h:5433
GridBlindDataSemantic
Blind-data Semantics that are currently understood by NanoVDB.
Definition: NanoVDB.h:424
bool isRootNext() const
return true if RootData is layout out immediately after TreeData in memory
Definition: NanoVDB.h:3966
bool operator==(const BaseBBox &rhs) const
Definition: NanoVDB.h:2235
uint32_t packed
Definition: NanoVDB.h:1864
Version mVersion
Definition: NanoVDB.h:3516
void setOn(uint32_t offset)
Definition: NanoVDB.h:5863
void setBitOff(std::initializer_list< uint8_t > list)
Definition: NanoVDB.h:2757
ConstValueIterator cbeginValueAll() const
Definition: NanoVDB.h:4481
GridFlags
Grid flags which indicate what extra information is present in the grid buffer.
Definition: NanoVDB.h:387
const NodeTrait< TreeT, LEVEL >::type * getNode() const
Definition: NanoVDB.h:7240
static uint32_t valueCount()
Definition: NanoVDB.h:5878
static uint32_t FindLowestOn(uint32_t v)
Returns the index of the lowest, i.e. least significant, on bit in the specified 32 bit word...
Definition: NanoVDB.h:2544
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:4044
uint32_t nameSize
Definition: NanoVDB.h:7881
CoordType getOrigin() const
Definition: NanoVDB.h:5144
ChildIter()
Definition: NanoVDB.h:5002
ReadAccessor< ValueT, LEVEL0, LEVEL1, LEVEL2 > createAccessor(const NanoGrid< ValueT > &grid)
Free-standing function for convenient creation of a ReadAccessor with optional and customizable node ...
Definition: NanoVDB.h:7487
RootT RootType
Definition: NanoVDB.h:3998
Definition: NanoVDB.h:247
float type
Definition: NanoVDB.h:703
CoordBBox bbox
Definition: NanoVDB.h:8200
bool isEmpty() const
Definition: NanoVDB.h:3960
float Type
Definition: NanoVDB.h:709
Visits all tile values and child nodes of this node.
Definition: NanoVDB.h:5111
GridType mGridType
Definition: NanoVDB.h:3526
Vec3 & operator-=(const Vec3 &v)
Definition: NanoVDB.h:1611
ChildNodeType * probeChild(const CoordType &ijk)
Definition: NanoVDB.h:5250
uint64_t mChecksum
Definition: NanoVDB.h:3515
uint32_t nodeCount(int level) const
Definition: NanoVDB.h:4085
ChannelAccessor(const NanoGrid< IndexT > &grid, ChannelT *channelPtr)
Ctor from an IndexGrid and an external channel.
Definition: NanoVDB.h:7749
Definition: NanoVDB.h:2858
Iterator end() const
Definition: NanoVDB.h:2421
NodeTrait< RootT, 1 >::type * getFirstLower()
Definition: NanoVDB.h:4139
GridType gridType
Definition: NanoVDB.h:7876
Define static boolean tests for template build types.
Definition: NanoVDB.h:471
T dot(const Vec4T &v) const
Definition: NanoVDB.h:1771
uint32_t hash() const
Return a hash key derived from the existing coordinates.
Definition: NanoVDB.h:1507
Bit-mask to encode active states and facilitate sequential iterators and a fast codec for I/O compres...
Definition: NanoVDB.h:2824
int32_t & x()
Definition: NanoVDB.h:1335
uint64_t pointCount() const
Definition: NanoVDB.h:5987
uint64_t getValue(uint32_t i) const
Definition: NanoVDB.h:5990
T * asPointer()
return a non-const raw constant pointer to array of three vector components
Definition: NanoVDB.h:1687
BBox< Vec3d > worldBBox
Definition: NanoVDB.h:7878
bool operator!=(const BaseBBox &rhs) const
Definition: NanoVDB.h:2236
void clear()
Reset this access to its initial state, i.e. with an empty cache Noop since this template specializa...
Definition: NanoVDB.h:6595
static size_t memUsage()
Definition: NanoVDB.h:1343
CoordT CoordType
Definition: NanoVDB.h:6709
void setOff()
Set all bits off.
Definition: NanoVDB.h:3046
static ElementType scalar(const T &v)
Definition: NanoVDB.h:1987
bool isValid(const GridBlindDataClass &blindClass, const GridBlindDataSemantic &blindSemantics, const GridType &blindType)
return true if the combination of GridBlindDataClass, GridBlindDataSemantic and GridType is valid...
Definition: NanoVDB.h:906
bool isChild() const
Definition: NanoVDB.h:4240
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:7290
NodeT & operator*() const
Definition: NanoVDB.h:4402
void localToGlobalCoord(Coord &ijk) const
modifies local coordinates to global coordinates of a tile or child node
Definition: NanoVDB.h:5278
void toggle()
brief Toggle the state of all bits in the mask
Definition: NanoVDB.h:3060
uint64_t mData2
Definition: NanoVDB.h:3530
const Vec3T & max() const
Definition: NanoVDB.h:2242
ChannelT & operator()(int i, int j, int k) const
Definition: NanoVDB.h:7793
Vec3T applyIJTF(const Vec3T &xyz) const
Definition: NanoVDB.h:3607
const NanoGrid< BuildT > & grid() const
Definition: NanoVDB.h:7612
typename ChildT::ValueType ValueT
Definition: NanoVDB.h:4176
float mMinimum
Definition: NanoVDB.h:5516
BBox< Coord > CoordBBox
Definition: NanoVDB.h:2535
void clear()
Reset this access to its initial state, i.e. with an empty cache.
Definition: NanoVDB.h:6924
Vec3T applyJacobianF(const Vec3T &xyz) const
Definition: NanoVDB.h:3603
uint8_t bitWidth() const
Definition: NanoVDB.h:5686
DenseIterator beginAll() const
Definition: NanoVDB.h:2926
Version(uint32_t data)
Definition: NanoVDB.h:957
static constexpr uint8_t bitWidth()
Definition: NanoVDB.h:5657
typename NanoLeaf< BuildT >::FloatType FloatType
Definition: NanoVDB.h:8194
bool isOff() const
Definition: NanoVDB.h:2786
void setMax(const ValueType &)
Definition: NanoVDB.h:5819
T Pow4(T x)
Definition: NanoVDB.h:1180
uint32_t totalNodeCount() const
Definition: NanoVDB.h:4091
Definition: NanoVDB.h:5837
ValueType getValue(uint32_t offset) const
Return the voxel value at the given offset.
Definition: NanoVDB.h:6251
KeyT key
Definition: NanoVDB.h:4244
uint64_t FloatType
Definition: NanoVDB.h:2025
bool isCached(const CoordType &ijk) const
Definition: NanoVDB.h:7277
typename DataType::StatsT FloatType
Definition: NanoVDB.h:4973
bool ValueType
Definition: NanoVDB.h:5742
A simple vector class with three components, similar to openvdb::math::Vec3.
Definition: NanoVDB.h:1298
Mask & operator^=(const Mask &other)
Bitwise XOR.
Definition: NanoVDB.h:3096
bool isActive() const
Return true if this node or any of its child nodes contain active values.
Definition: NanoVDB.h:5292
Dummy type for a voxel whose value equals an offset into an external value array of active values...
Definition: NanoVDB.h:257
const Map & map() const
Definition: NanoVDB.h:7574
Top-most node of the VDB tree structure.
Definition: NanoVDB.h:4329
void setValue(uint32_t offset, bool v)
Definition: NanoVDB.h:5763
int64_t child
Definition: NanoVDB.h:4870
bool operator<=(const Coord &rhs) const
Return true if this Coord is lexicographically less or equal to the given Coord.
Definition: NanoVDB.h:1384
#define NANOVDB_MAJOR_VERSION_NUMBER
Definition: NanoVDB.h:133
bool probeValue(const CoordT &ijk, ValueType &v) const
Return true if the voxel value at the given coordinate is active and updates v with the value...
Definition: NanoVDB.h:6287
Version version() const
Definition: NanoVDB.h:7585
uint8_t ArrayType
Definition: NanoVDB.h:5579
Vec4 operator/(const T &s) const
Definition: NanoVDB.h:1783
Struct to derive node type from its level in a given grid, tree or root while preserving constness...
Definition: NanoVDB.h:3404
ValueType operator*() const
Definition: NanoVDB.h:5094
typename GridT::TreeType type
Definition: NanoVDB.h:3976
T length() const
Definition: NanoVDB.h:1587
uint64_t getMax() const
Definition: NanoVDB.h:5909
Vec3T applyJacobian(const Vec3T &ijk) const
Apply the linear forward 3x3 transformation to an input 3d vector using 64bit floating point arithmet...
Definition: NanoVDB.h:3229
const ChildT * probeChild(ValueType &value) const
Definition: NanoVDB.h:5128
Definition: NanoVDB.h:4384
uint32_t level
Definition: NanoVDB.h:8197
uint16_t padding
Definition: NanoVDB.h:7885
static CoordT KeyToCoord(const KeyT &key)
Definition: NanoVDB.h:4194
Vec3< float > asVec3s() const
Return a single precision floating-point vector of this coordinate.
Definition: NanoVDB.h:1712
uint32_t mTileCount[3]
Definition: NanoVDB.h:3944
typename RootT::ChildNodeType Node2
Definition: NanoVDB.h:4009
Vec3 & minComponent(const Vec3 &other)
Perform a component-wise minimum with the other Coord.
Definition: NanoVDB.h:1635
ValueType mMinimum
Definition: NanoVDB.h:5448
DenseIterator(uint32_t pos=Mask::SIZE)
Definition: NanoVDB.h:2895
uint64_t Type
Definition: NanoVDB.h:660
uint64_t type
Definition: NanoVDB.h:668
bool isPointIndex() const
Definition: NanoVDB.h:7559
T type
Definition: NanoVDB.h:601
ValueT mMaximum
Definition: NanoVDB.h:4884
BBox(const CoordT &min, const CoordT &max)
Definition: NanoVDB.h:2426
Mask & operator=(const Mask &other)
Definition: NanoVDB.h:2965
void setValueOnly(uint32_t offset, const ValueType &value)
Definition: NanoVDB.h:5466
const NanoGrid< Point > & grid() const
Definition: NanoVDB.h:7679
bool isActive() const
Definition: NanoVDB.h:6142
Vec3T applyInverseMap(const Vec3T &xyz) const
Apply the inverse affine mapping to a vector using 64bit floating point arithmetics.
Definition: NanoVDB.h:3246
const LeafNodeType * getFirstLeaf() const
Definition: NanoVDB.h:4138
static bool safeCast(const NanoGrid< T > &grid)
return true if it is safe to cast the grid to a pointer of type GridMetaData, i.e. construction can be avoided.
Definition: NanoVDB.h:7552
Vec3T & max()
Definition: NanoVDB.h:2240
T ElementType
Definition: NanoVDB.h:1975
Vec4 & operator+=(const Vec4 &v)
Definition: NanoVDB.h:1784
Coord & maxComponent(const Coord &other)
Perform a component-wise maximum with the other Coord.
Definition: NanoVDB.h:1455
typename RootType::LeafNodeType LeafNodeType
Definition: NanoVDB.h:3706
float Type
Definition: NanoVDB.h:723
uint32_t getPatch() const
Definition: NanoVDB.h:973
bool hasStdDeviation() const
Definition: NanoVDB.h:3841
uint64_t Type
Definition: NanoVDB.h:667
Struct with all the member data of the RootNode (useful during serialization of an openvdb RootNode) ...
Definition: NanoVDB.h:4174
Data encoded at the head of each segment of a file or stream.
Definition: NanoVDB.h:7848
const GridBlindMetaData & blindMetaData(uint32_t n) const
Definition: NanoVDB.h:3903
openvdb::GridBase Grid
Definition: Utils.h:34
static BBox createCube(const Coord &min, typename Coord::ValueType dim)
Definition: NanoVDB.h:2324
uint8_t * treePtr()
Definition: NanoVDB.h:3610
const uint32_t & activeTileCount(uint32_t level) const
Definition: NanoVDB.h:7580
static T * alignPtr(T *p)
offset the specified pointer so it is aligned.
Definition: NanoVDB.h:757
int32_t Floor(float x)
Definition: NanoVDB.h:1149
const ValueType & maximum() const
Return a const reference to the maximum active value encoded in this root node and any of its child n...
Definition: NanoVDB.h:4610
Mask()
Initialize all bits to zero.
Definition: NanoVDB.h:2929
ReadAccessor(const TreeT &tree)
Constructor from a tree.
Definition: NanoVDB.h:6588
ValueIterator beginValue() const
Definition: NanoVDB.h:5073
bool isMaskOn(uint32_t offset) const
Definition: NanoVDB.h:5950
typename RootNodeType::ChildNodeType UpperNodeType
Definition: NanoVDB.h:4000
ConstValueOnIterator cbeginValueOn() const
Definition: NanoVDB.h:4525
double FloatType
Definition: NanoVDB.h:2001
float getValue(uint32_t i) const
Definition: NanoVDB.h:5689
Version version
Definition: NanoVDB.h:7886
const BBox< Coord > & indexBBox() const
Definition: NanoVDB.h:7576
void setMax(const ValueType &)
Definition: NanoVDB.h:6005
uint64_t type
Definition: NanoVDB.h:675
static DstT * PtrAdd(SrcT *p, int64_t offset)
Adds a byte offset of a non-const pointer to produce another non-const pointer.
Definition: NanoVDB.h:795
T type
Definition: NanoVDB.h:587
ValueIterator()
Definition: NanoVDB.h:5044
Vec4 operator-() const
Definition: NanoVDB.h:1777
Delta for small floating-point offsets.
Definition: NanoVDB.h:1035
GridClass mGridClass
Definition: NanoVDB.h:3525
Dummy type for a voxel whose value equals an offset into an external value array. ...
Definition: NanoVDB.h:252
Maps one type (e.g. the build types above) to other (actual) types.
Definition: NanoVDB.h:651
Coord()
Initialize all coordinates to zero.
Definition: NanoVDB.h:1309
Vec3T applyInverseJacobianF(const Vec3T &xyz) const
Definition: NanoVDB.h:3605
Vec3T applyIJT(const Vec3T &xyz) const
Apply the transposed inverse 3x3 transformation to an input 3d vector using 64bit floating point arit...
Definition: NanoVDB.h:3287
ValueType operator()(int i, int j, int k) const
Definition: NanoVDB.h:6989
DataType * data()
Definition: NanoVDB.h:6173
typename RootT::ValueType ValueType
Definition: NanoVDB.h:6558
Vec3d getVoxelSize() const
Return a voxels size in each coordinate direction, measured at the origin.
Definition: NanoVDB.h:3292
ValueType getValue(const CoordType &ijk) const
Definition: NanoVDB.h:6757
bool operator<=(const Version &rhs) const
Definition: NanoVDB.h:967
ValueOnIter(RootT *parent)
Definition: NanoVDB.h:4493
Vec3 & operator+=(const Coord &ijk)
Definition: NanoVDB.h:1604
void setMax(float max)
Definition: NanoVDB.h:5556
double ValueType
Definition: NanoVDB.h:1536
const FloatType & stdDeviation() const
Return a const reference to the standard deviation of all the active values encoded in this root node...
Definition: NanoVDB.h:4619
typename ChildT::CoordType CoordType
Definition: NanoVDB.h:4343
Definition: NanoVDB.h:4484
void setAvg(const FloatType &)
Definition: NanoVDB.h:5861
void setChild(uint32_t n, const void *ptr)
Definition: NanoVDB.h:4900
const uint8_t * nodePtr() const
Return a non-const uint8_t pointer to the first node at LEVEL.
Definition: NanoVDB.h:3621
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:6987
Vec3 operator-(const Vec3 &v) const
Definition: NanoVDB.h:1592
void setMax(const ValueT &v)
Definition: NanoVDB.h:4952
Vec3T indexToWorldGrad(const Vec3T &grad) const
transform the gradient from index space to world space.
Definition: NanoVDB.h:3788
static uint32_t padding()
Definition: NanoVDB.h:6239
bool getAvg() const
Definition: NanoVDB.h:5814
DenseIter operator++(int)
Definition: NanoVDB.h:4566
const typename GridOrTreeOrRootT::LeafNodeType type
Definition: NanoVDB.h:3419
float length() const
Definition: NanoVDB.h:1938
static size_t memUsage()
Return memory usage in bytes for the class.
Definition: NanoVDB.h:5169
static Coord Floor(const Vec3T &xyz)
Return the largest integer coordinates that are not greater than xyz (node centered conversion)...
Definition: NanoVDB.h:1499
bool isApproxZero(const Type &x)
Definition: NanoVDB.h:1083
void setOn(uint32_t offset)
Definition: NanoVDB.h:5997
const NodeTrait< RootT, LEVEL >::type * getFirstNode() const
return a const pointer to the first node of the specified level
Definition: NanoVDB.h:4131
Rgba8(const Vec4f &rgba)
Vec4f r,g,b,a ctor.
Definition: NanoVDB.h:1925
const BBoxType & bbox() const
Return a const reference to the index bounding box of all the active values in this tree...
Definition: NanoVDB.h:4594
Coord operator-(const Coord &rhs) const
Definition: NanoVDB.h:1425
const Map & map() const
Return a const reference to the Map for this grid.
Definition: NanoVDB.h:3765
static double value()
Definition: NanoVDB.h:1028
Definition: NanoVDB.h:2295
bool getValue(uint32_t i) const
Definition: NanoVDB.h:5758
ValueType getValue(const CoordType &ijk) const
Definition: NanoVDB.h:6983
FloatType getAvg() const
Definition: NanoVDB.h:6001
uint8_t octant() const
Return the octant of this Coord.
Definition: NanoVDB.h:1511
bool isValid() const
Definition: NanoVDB.h:7853
ValueType operator*() const
Definition: NanoVDB.h:4451
bool isValid() const
Methods related to the classification of this grid.
Definition: NanoVDB.h:3826
uint16_t gridCount
Definition: NanoVDB.h:7851
Index64 memUsage(const TreeT &tree, bool threaded=true)
Return the total amount of memory in bytes occupied by this tree.
Definition: Count.h:493
CoordType getCoord() const
Definition: NanoVDB.h:5028
void setMax(const ValueType &)
Definition: NanoVDB.h:5860
DenseIter & operator++()
Definition: NanoVDB.h:4560
T Abs(T x)
Definition: NanoVDB.h:1185
bool isOn(uint32_t n) const
Return true if the given bit is set.
Definition: NanoVDB.h:2983
uint64_t volume() const
Definition: NanoVDB.h:2463
Vec3T applyMap(const Vec3T &ijk) const
Apply the forward affine transformation to a vector using 64bit floating point arithmetics.
Definition: NanoVDB.h:3212
bool operator!=(const Vec4 &rhs) const
Definition: NanoVDB.h:1756
typename Node2::ChildNodeType Node1
Definition: NanoVDB.h:4010
bool probeValue(const CoordType &ijk, ValueType &v) const
Definition: NanoVDB.h:7295
Dummy type for a 16 bit floating point values.
Definition: NanoVDB.h:277
bool isValid() const
return true if this meta data has a valid combination of semantic, class and value tags ...
Definition: NanoVDB.h:3365
RootT RootNodeType
Definition: NanoVDB.h:3999
const uint8_t & b() const
Definition: NanoVDB.h:1947
uint32_t id() const
Definition: NanoVDB.h:970
#define NANOVDB_MAGIC_NUMBER
Definition: NanoVDB.h:126
const char * c_str() const
Definition: NanoVDB.h:981
bool getDev() const
Definition: NanoVDB.h:5762
Vec4 & operator=(const Vec4T< T2 > &rhs)
Definition: NanoVDB.h:1758
float getValue(uint32_t i) const
Definition: NanoVDB.h:5658
bool isEmpty() const
Return true if this RootNode is empty, i.e. contains no values or nodes.
Definition: NanoVDB.h:4628
float Fract(float x)
Definition: NanoVDB.h:1140
DenseIter()
Definition: NanoVDB.h:4534
NodeTrait< RootT, 2 >::type * getFirstUpper()
Definition: NanoVDB.h:4141
uint8_t & g()
Definition: NanoVDB.h:1950
void fill(const ValueType &v)
Definition: NanoVDB.h:5487
Vec4 operator*(const T &s) const
Definition: NanoVDB.h:1782
BitFlags & operator=(Type n)
required for backwards compatibility
Definition: NanoVDB.h:2812
ValueType getValue(const CoordType &ijk) const
Definition: NanoVDB.h:7286
void ArrayType
Definition: NanoVDB.h:5843
const ChildNodeType * probeChild(const CoordType &ijk) const
Definition: NanoVDB.h:4684
Coord & operator>>=(uint32_t n)
Definition: NanoVDB.h:1410
static uint64_t memUsage()
Definition: NanoVDB.h:5855
Definition: NanoVDB.h:3330
int64_t child
Definition: NanoVDB.h:4245
bool empty() const
Return true if this bounding box is empty, e.g. uninitialized.
Definition: NanoVDB.h:2455
OnIterator beginOn() const
Definition: NanoVDB.h:2922
BuildT ArrayType
Definition: NanoVDB.h:5440
uint32_t mBlindMetadataCount
Definition: NanoVDB.h:3528
Vec3T indexToWorldDir(const Vec3T &dir) const
transformation from index space direction to world space direction
Definition: NanoVDB.h:3778
BuildT BuildType
Definition: NanoVDB.h:6707
Version version
Definition: NanoVDB.h:7850
void setOn(uint32_t offset)
Definition: NanoVDB.h:5768
uint32_t CountOn(uint64_t v)
Definition: NanoVDB.h:2662
bool isMaskOn(uint32_t offset) const
Definition: NanoVDB.h:5939
uint64_t blindDataSize() const
return size in bytes of the blind data represented by this blind meta data
Definition: NanoVDB.h:3393
Vec3(T x, T y, T z)
Definition: NanoVDB.h:1542
bool BuildType
Definition: NanoVDB.h:5743
8-bit red, green, blue, alpha packed into 32 bit unsigned int
Definition: NanoVDB.h:1859
uint64_t activeVoxelCount() const
Definition: NanoVDB.h:7579
const ValueT & getMin() const
Definition: NanoVDB.h:4942
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:4634
ValueOnIter()
Definition: NanoVDB.h:4489
typename UpperNodeType::ChildNodeType LowerNodeType
Definition: NanoVDB.h:3705
T length() const
Definition: NanoVDB.h:1776
Version(uint32_t major, uint32_t minor, uint32_t patch)
Definition: NanoVDB.h:958
TileT * tile() const
Definition: NanoVDB.h:4370
VecT< GridHandleT > readUncompressedGrids(const char *fileName, const typename GridHandleT::BufferType &buffer=typename GridHandleT::BufferType())
Read a multiple un-compressed NanoVDB grids from a file and return them as a vector.
Definition: NanoVDB.h:8024
uint64_t Type
Definition: NanoVDB.h:730
AccessorType getAccessor() const
Return a new instance of a ReadAccessor used to access values in this grid.
Definition: NanoVDB.h:3759
static constexpr uint32_t padding()
Return padding of this class in bytes, due to aliasing and 32B alignment.
Definition: NanoVDB.h:5457
CoordT mBBoxMin
Definition: NanoVDB.h:5846
const FloatType & average() const
Return a const reference to the average of all the active values encoded in this internal node and an...
Definition: NanoVDB.h:5189
CoordT getCoord() const
Definition: NanoVDB.h:6103
CoordT CoordType
Definition: NanoVDB.h:6892
Dummy type for a variable bit quantization of floating point values.
Definition: NanoVDB.h:297
const BBox< CoordType > & bbox() const
Return a const reference to the bounding box in index space of active values in this internal node an...
Definition: NanoVDB.h:5198
ValueOnIterator(const InternalNode *parent)
Definition: NanoVDB.h:5088
bool operator<(const Version &rhs) const
Definition: NanoVDB.h:966
int MaxIndex(const Vec3T &v)
Definition: NanoVDB.h:1268
DenseIterator cbeginChildAll() const
Definition: NanoVDB.h:5153
uint32_t valueCount() const
Definition: NanoVDB.h:5903
const MaskType< LOG2DIM > & getValueMask() const
Definition: NanoVDB.h:5173
StatsT mAverage
Definition: NanoVDB.h:4212
ValueIterator & operator++()
Definition: NanoVDB.h:6148
const NanoGrid< IndexT > & grid() const
Return a const reference to the IndexGrid.
Definition: NanoVDB.h:7762
Visits all values in a leaf node, i.e. both active and inactive values.
Definition: NanoVDB.h:6114
Vec3T applyIJT(const Vec3T &xyz) const
Definition: NanoVDB.h:3596
Vec3 & operator/=(const T &s)
Definition: NanoVDB.h:1632
Rgba8(uint8_t r, uint8_t g, uint8_t b, uint8_t a=255u)
integer r,g,b,a ctor where alpha channel defaults to opaque
Definition: NanoVDB.h:1894
Type & data()
Definition: NanoVDB.h:2728
T type
Definition: NanoVDB.h:499
Visits active tile values of this node only.
Definition: NanoVDB.h:5077
void setOrigin(const T &ijk)
Definition: NanoVDB.h:5775
int32_t & z()
Definition: NanoVDB.h:1337
const uint8_t & g() const
Definition: NanoVDB.h:1946
Visits all inactive values in a leaf node.
Definition: NanoVDB.h:6081
uint8_t flags() const
Definition: NanoVDB.h:6196
bool isFloatingPoint(GridType gridType)
return true if the GridType maps to a floating point type
Definition: NanoVDB.h:817
Coord & operator+=(int n)
Definition: NanoVDB.h:1417
Vec4 operator/(const Vec4 &v) const
Definition: NanoVDB.h:1779
Like ValueIndex but with a mutable mask.
Definition: NanoVDB.h:262
const GridType & gridType() const
Definition: NanoVDB.h:3827
typename RootT::ValueType ValueType
Definition: NanoVDB.h:4003
const ValueType & minimum() const
Return a const reference to the minimum active value encoded in this root node and any of its child n...
Definition: NanoVDB.h:4607
ValueOffIterator cbeginValueOff() const
Definition: NanoVDB.h:6111
GridData & operator=(const GridData &other)
Use this method to initiate most member dat.
Definition: NanoVDB.h:3532
Mask & operator-=(const Mask &other)
Bitwise difference.
Definition: NanoVDB.h:3087
Definition: NanoVDB.h:3473
NodeT * operator->() const
Definition: NanoVDB.h:4407
ChildIterator beginChild()
Definition: NanoVDB.h:4431
GridMetaData(const NanoGrid< T > &grid)
Definition: NanoVDB.h:7519
float FloatType
Definition: NanoVDB.h:1995
void setMask(MaskT mask, bool on)
Definition: NanoVDB.h:2783
ChannelT * setChannel(ChannelT *channelPtr)
Change to an external channel.
Definition: NanoVDB.h:7775
Coord offsetToGlobalCoord(uint32_t n) const
Definition: NanoVDB.h:5284
uint32_t IndexType
Definition: NanoVDB.h:1306
bool isUnknown() const
Definition: NanoVDB.h:7563
Iterator()
Definition: NanoVDB.h:2861
typename RootNodeType::ChildNodeType UpperNodeType
Definition: NanoVDB.h:3704
bool getMax() const
Definition: NanoVDB.h:5813
auto getNodeInfo(const CoordType &ijk) const
Definition: NanoVDB.h:6611
typename ChildT::BuildType BuildT
Definition: NanoVDB.h:4177
bool isGridIndex() const
Definition: NanoVDB.h:3833
ValueType getValue(uint32_t i) const
Definition: NanoVDB.h:5465
ValueType getMax() const
Definition: NanoVDB.h:5475
typename BuildT::ValueType ValueType
Definition: NanoVDB.h:3708
bool isStaggered() const
Definition: NanoVDB.h:7558
uint64_t voxelPoints(const Coord &ijk, const AttT *&begin, const AttT *&end) const
get iterators over attributes to points at a specific voxel location
Definition: NanoVDB.h:7638
Vec3 & operator-=(const Coord &ijk)
Definition: NanoVDB.h:1618
float ValueType
Definition: NanoVDB.h:5508
uint64_t lastOffset() const
Definition: NanoVDB.h:5907
CoordT Round(const Vec3T< RealT > &xyz)
const uint32_t & activeTileCount(uint32_t level) const
Return the total number of active tiles at the specified level of the tree.
Definition: NanoVDB.h:4072
uint64_t getValue(uint32_t i) const
Definition: NanoVDB.h:5912
void setOrigin(const T &ijk)
Definition: NanoVDB.h:5485
typename ChildT::template MaskType< LOG2DIM > MaskT
Definition: NanoVDB.h:4864
BitFlags< 32 > mFlags
Definition: NanoVDB.h:3517
CoordType getOrigin() const
Definition: NanoVDB.h:5023
uint32_t pos() const
Definition: NanoVDB.h:2901
uint8_t mFlags
Definition: NanoVDB.h:5969
void setMask(uint32_t offset, bool v)
Definition: NanoVDB.h:5951
static uint32_t wordCount()
Return the number of machine words used by this Mask.
Definition: NanoVDB.h:2837
BBox()
Definition: NanoVDB.h:2422
bool operator<(const Iterator &rhs) const
Definition: NanoVDB.h:2406
CoordType getCoord() const
Definition: NanoVDB.h:5149
void writeUncompressedGrid(StreamT &os, const GridData *gridData, bool raw=false)
This is a standalone alternative to io::writeGrid(...,Codec::NONE) defined in util/IO.h Unlike the latter this function has no dependencies at all, not even NanoVDB.h, so it also works if client code only includes PNanoVDB.h!
Definition: NanoVDB.h:7921
enable_if< BuildTraits< T >::is_index, const uint64_t & >::type valueCount() const
Return the total number of values indexed by this IndexGrid.
Definition: NanoVDB.h:3743
ValueType max() const
Return the largest vector component.
Definition: NanoVDB.h:1663
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5514
bool isInteger(GridType gridType)
Return true if the GridType maps to a POD integer type.
Definition: NanoVDB.h:843
int64_t mBlindMetadataOffset
Definition: NanoVDB.h:3527
ValueType getMin() const
Definition: NanoVDB.h:5474
Coord offsetBy(ValueType dx, ValueType dy, ValueType dz) const
Definition: NanoVDB.h:1482
float mTaperF
Definition: NanoVDB.h:3162
Implements Tree::probeLeaf(Coord)
Definition: NanoVDB.h:3481
T type
Definition: NanoVDB.h:581
void setAvg(const FloatType &v)
Definition: NanoVDB.h:5481
ReadAccessor(const RootT &root)
Constructor from a root node.
Definition: NanoVDB.h:7198
uint32_t getDim(const CoordType &ijk, const RayT &ray) const
Definition: NanoVDB.h:6817
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5849
Vec4 operator+(const Vec4 &v) const
Definition: NanoVDB.h:1780
Type data() const
Definition: NanoVDB.h:2727
void setAvg(const StatsT &v)
Definition: NanoVDB.h:4953
uint32_t getDim(const CoordType &ijk, const RayT &ray) const
Definition: NanoVDB.h:7084
T type
Definition: NanoVDB.h:572
typename RootT::CoordType CoordType
Definition: NanoVDB.h:6559
ReadAccessor(const RootT &root)
Constructor from a root node.
Definition: NanoVDB.h:6576
GridClass gridClass
Definition: NanoVDB.h:7877
Codec codec
Definition: NanoVDB.h:7852
ChildIter()
Definition: NanoVDB.h:4391
void next()
Definition: NanoVDB.h:4369
BBox(const BaseBBox< Coord > &bbox)
Definition: NanoVDB.h:2329
RootType RootNodeType
Definition: NanoVDB.h:3703
Vec4 & minComponent(const Vec4 &other)
Perform a component-wise minimum with the other Coord.
Definition: NanoVDB.h:1811
uint64_t gridSize
Definition: NanoVDB.h:7875
BitFlags(std::initializer_list< MaskT > list)
Definition: NanoVDB.h:2722
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:7294
CoordT mBBoxMin
Definition: NanoVDB.h:5748
static CoordT OffsetToLocalCoord(uint32_t n)
Compute the local coordinates from a linear offset.
Definition: NanoVDB.h:6204
const typename remove_const< T >::type type
Definition: NanoVDB.h:624
T & operator[](int i)
Definition: NanoVDB.h:1573
uint64_t getDev() const
Definition: NanoVDB.h:5911
bool isOff(uint32_t n) const
Return true if the given bit is NOT set.
Definition: NanoVDB.h:2986
typename remove_const< T >::type type
Definition: NanoVDB.h:613
Vec3T applyMap(const Vec3T &xyz) const
Definition: NanoVDB.h:3588
Vec3T & operator[](int i)
Definition: NanoVDB.h:2238
uint32_t pos() const
Definition: NanoVDB.h:4368
const void * blindData(uint32_t n) const
Returns a const pointer to the blindData at the specified linear offset.
Definition: NanoVDB.h:3882
void setValueOnly(uint32_t offset, const ValueType &v)
Sets the value at the specified location but leaves its state unchanged.
Definition: NanoVDB.h:6269
FloatType variance() const
Return the variance of all the active values encoded in this internal node and any of its child nodes...
Definition: NanoVDB.h:5192
uint64_t voxelPoints(const Coord &ijk, const AttT *&begin, const AttT *&end) const
get iterators over attributes to points at a specific voxel location
Definition: NanoVDB.h:7704
bool isCached1(const CoordType &ijk) const
Definition: NanoVDB.h:6968
Vec3 operator/(const T &s) const
Definition: NanoVDB.h:1596
float lengthSqr() const
Definition: NanoVDB.h:1932
void setMax(const bool &)
Definition: NanoVDB.h:5770
typename ChildT::BuildType BuildT
Definition: NanoVDB.h:4861
const DataType * data() const
Definition: NanoVDB.h:5163
const uint8_t & operator[](int n) const
Definition: NanoVDB.h:1941
void setLongGridNameOn(bool on=true)
Definition: NanoVDB.h:3574
Coord round() const
Definition: NanoVDB.h:1522
static uint64_t memUsage()
Definition: NanoVDB.h:4898
ValueOnIter & operator++()
Definition: NanoVDB.h:4505
void setDev(const FloatType &v)
Definition: NanoVDB.h:5482
Iterator(uint32_t pos, const Mask *parent)
Definition: NanoVDB.h:2866
bool hasOverlap(const BBox &b) const
Return true if the given bounding box overlaps with this bounding box.
Definition: NanoVDB.h:2476
uint64_t mPointCount
Definition: NanoVDB.h:5973
T type
Definition: NanoVDB.h:595
const ValueT & getMax() const
Definition: NanoVDB.h:4309
ReadAccessor(const GridT &grid)
Constructor from a grid.
Definition: NanoVDB.h:6912
const LeafT * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:6614
void setDev(const bool &)
Definition: NanoVDB.h:5772
uint64_t gridSize() const
Definition: NanoVDB.h:7570
MaskT< LOG2DIM > mMask
Definition: NanoVDB.h:5948
DataType * data()
Definition: NanoVDB.h:3722
const GridClass & gridClass() const
Definition: NanoVDB.h:7555
ValueType operator()(const CoordType &ijk) const
Definition: NanoVDB.h:6762
PointAccessor(const NanoGrid< BuildT > &grid)
Definition: NanoVDB.h:7599
Vec3T mCoord[2]
Definition: NanoVDB.h:2234
const LeafT * probeLeaf(const CoordType &ijk) const
Definition: NanoVDB.h:6767
ValueT mMinimum
Definition: NanoVDB.h:4883
RootT & root()
Definition: NanoVDB.h:4026
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5751
ValueOnIterator beginValueOn()
Definition: NanoVDB.h:4524
const StatsT & average() const
Definition: NanoVDB.h:4310
constexpr T pi()
Pi constant taken from Boost to match old behaviour.
Definition: NanoVDB.h:995
BaseBBox & expand(const BaseBBox &bbox)
Expand this bounding box to enclose the given bounding box.
Definition: NanoVDB.h:2258
ValueIterator(const LeafNode *parent)
Definition: NanoVDB.h:6125
uint64_t ValueType
Definition: NanoVDB.h:5961
void setMaskOn(std::initializer_list< MaskT > list)
Definition: NanoVDB.h:2769
ChannelAccessor(const NanoGrid< IndexT > &grid, uint32_t channelID=0u)
Ctor from an IndexGrid and an integer ID of an internal channel that is assumed to exist as blind dat...
Definition: NanoVDB.h:7738
Dummy type for a 8bit quantization of float point values.
Definition: NanoVDB.h:287
typename NanoLeaf< BuildT >::ValueType ValueType
Definition: NanoVDB.h:8193
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:6612
uint64_t leafPoints(const Coord &ijk, const AttT *&begin, const AttT *&end) const
Return the number of points in the leaf node containing the coordinate ijk. If this return value is l...
Definition: NanoVDB.h:7626
uint32_t pos() const
Definition: NanoVDB.h:2873
Maximum floating-point values.
Definition: NanoVDB.h:1051
bool operator==(const Mask &other) const
Definition: NanoVDB.h:2971
void setOrigin(const T &ijk)
Definition: NanoVDB.h:5565
bool hasMinMax() const
Definition: NanoVDB.h:3837
bool operator==(const Iterator &rhs) const
Definition: NanoVDB.h:2396
Dummy type for a voxel whose value equals its binary active state.
Definition: NanoVDB.h:272
uint8_t mFlags
Definition: NanoVDB.h:5848
uint64_t mPrefixSum
Definition: NanoVDB.h:5850
bool operator<(const Tuple< SIZE, T0 > &t0, const Tuple< SIZE, T1 > &t1)
Definition: Tuple.h:174
DenseIterator()
Definition: NanoVDB.h:5117
uint64_t getMax() const
Definition: NanoVDB.h:5883
Version()
Definition: NanoVDB.h:951
static constexpr uint32_t padding()
Definition: NanoVDB.h:5680
Rgba8(const Vec3f &rgb)
Vec3f r,g,b ctor (alpha channel it set to 1)
Definition: NanoVDB.h:1918
bool setGridName(const char *src)
Definition: NanoVDB.h:3577
int32_t & y()
Definition: NanoVDB.h:1336
Vec3 & operator*=(const T &s)
Definition: NanoVDB.h:1625
uint32_t mGridIndex
Definition: NanoVDB.h:3518
static uint64_t memUsage()
Definition: NanoVDB.h:5984
int findBlindDataForSemantic(GridBlindDataSemantic semantic) const
Return the index of the first blind data with specified semantic if found, otherwise -1...
Definition: NanoVDB.h:3910
uint64_t mVoxelCount
Definition: NanoVDB.h:3945
Vec3d voxelSize
Definition: NanoVDB.h:7880
const char * gridName() const
Return a c-string with the name of this grid.
Definition: NanoVDB.h:3858
bool operator!=(const Vec3 &rhs) const
Definition: NanoVDB.h:1562
Iterator(const BBox &b, const Coord &p)
Definition: NanoVDB.h:2371
uint64_t type
Definition: NanoVDB.h:661
void setBitOn(uint8_t bit)
Definition: NanoVDB.h:2749
static Coord OffsetToLocalCoord(uint32_t n)
Definition: NanoVDB.h:5270
static constexpr uint32_t padding()
Definition: NanoVDB.h:5651
uint64_t last(uint32_t i) const
Definition: NanoVDB.h:5989
bool isOn() const
Definition: NanoVDB.h:2785
const char * toStr(GridType gridType)
Maps a GridType to a c-string.
Definition: NanoVDB.h:349
uint32_t gridCount() const
Definition: NanoVDB.h:7572
GridBlindDataSemantic mSemantic
Definition: NanoVDB.h:3336
ValueOnIterator()
Definition: NanoVDB.h:5083
bool isValid() const
return true if the magic number and the version are both valid
Definition: NanoVDB.h:3568
CoordT CoordType
Definition: NanoVDB.h:7191
const RootT & root() const
Definition: NanoVDB.h:6597
float getMax() const
return the quantized maximum of the active values in this node
Definition: NanoVDB.h:5543
bool hasBBox() const
Definition: NanoVDB.h:3838
BaseBBox & translate(const Vec3T &xyz)
Definition: NanoVDB.h:2243
#define NANOVDB_ASSERT(x)
Definition: NanoVDB.h:190
T dot(const Vec3T &v) const
Definition: NanoVDB.h:1575
ValueT mBackground
Definition: NanoVDB.h:4209
const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType type
Definition: NanoVDB.h:3448
void clear()
Reset this access to its initial state, i.e. with an empty cache.
Definition: NanoVDB.h:6736
const NodeTrait< RootT, 2 >::type * getFirstUpper() const
Definition: NanoVDB.h:4142
BBox< CoordT > mBBox
Definition: NanoVDB.h:4878
const TreeType & tree() const
Return a const reference to the tree of the IndexGrid.
Definition: NanoVDB.h:7765
const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType type
Definition: NanoVDB.h:3434
Codec codec
Definition: NanoVDB.h:7884
uint64_t leafPoints(const Coord &ijk, const AttT *&begin, const AttT *&end) const
Return the number of points in the leaf node containing the coordinate ijk. If this return value is l...
Definition: NanoVDB.h:7693
float getMin() const
return the quantized minimum of the active values in this node
Definition: NanoVDB.h:5540
Defines an affine transform and its inverse represented as a 3x3 matrix and a vec3 translation...
Definition: NanoVDB.h:3157
T & getValue(const Coord &ijk, T *channelPtr) const
Return the value from a specified channel that maps to the specified coordinate.
Definition: NanoVDB.h:7807
ValueIter()
Definition: NanoVDB.h:4440
uint64_t FloatType
Definition: NanoVDB.h:5842
float Type
Definition: NanoVDB.h:702
uint32_t getDim(const CoordType &ijk, const RayT &ray) const
Definition: NanoVDB.h:6654
static double value()
Definition: NanoVDB.h:1044
const FloatType & stdDeviation() const
Return a const reference to the standard deviation of all the active values encoded in this internal ...
Definition: NanoVDB.h:5195
void setBit(uint8_t bit, bool on)
Definition: NanoVDB.h:2781
Coord ceil() const
Round each component if this Vec<T> down to its integer value.
Definition: NanoVDB.h:1672
typename GridOrTreeOrRootT::RootNodeType::ChildNodeType type
Definition: NanoVDB.h:3441
Coord(ValueType n)
Initializes all coordinates to the given signed integer.
Definition: NanoVDB.h:1315
uint32_t gridIndex() const
Return index of this grid in the buffer.
Definition: NanoVDB.h:3733
GridType
List of types that are currently supported by NanoVDB.
Definition: NanoVDB.h:317
ValueType operator()(const CoordType &ijk) const
Definition: NanoVDB.h:6988
ValueType operator()(int i, int j, int k) const
Definition: NanoVDB.h:6763
uint32_t mValueSize
Definition: NanoVDB.h:3335
void setMaskOff(std::initializer_list< MaskT > list)
Definition: NanoVDB.h:2775
typename BuildT::CoordType CoordType
Definition: NanoVDB.h:3710
uint32_t getMajor() const
Definition: NanoVDB.h:971
uint32_t findNext(uint32_t start) const
Definition: NanoVDB.h:3118
void setValue(const CoordT &ijk, const ValueType &v)
Sets the value at the specified location and activate its state.
Definition: NanoVDB.h:6264
Coord & operator&=(int n)
Definition: NanoVDB.h:1396
Class to access points at a specific voxel location.
Definition: NanoVDB.h:7592
bool isMaskOn(MaskT mask) const
Definition: NanoVDB.h:2790
ValueType getLastValue() const
If the last entry in this node&#39;s table is a tile, return the tile&#39;s value. Otherwise, return the result of calling getLastValue() on the child.
Definition: NanoVDB.h:5209
uint32_t blindDataCount() const
Return true if this grid is empty, i.e. contains no values or nodes.
Definition: NanoVDB.h:3870
void initMask(std::initializer_list< MaskT > list)
Definition: NanoVDB.h:2736
bool isUnknown() const
Definition: NanoVDB.h:3836
Vec4 & operator*=(const T &s)
Definition: NanoVDB.h:1800
ValueT mMaximum
Definition: NanoVDB.h:4211
bool isInside(const CoordT &p) const
Definition: NanoVDB.h:2468
const ChildT * getChild(const Tile *tile) const
Definition: NanoVDB.h:4302
bool getMin() const
Definition: NanoVDB.h:5812
const BlindDataT * getBlindData(uint32_t n) const
Definition: NanoVDB.h:3890
ValueT mMinimum
Definition: NanoVDB.h:4210
uint64_t offset() const
Definition: NanoVDB.h:5986
NodeTrait< RootT, LEVEL >::type * getFirstNode()
return a pointer to the first node at the specified level
Definition: NanoVDB.h:4121
const GridClass & gridClass() const
Definition: NanoVDB.h:3828
bool FloatType
Definition: NanoVDB.h:2037
uint32_t findPrev(uint32_t start) const
Definition: NanoVDB.h:3135
void setOn()
Definition: NanoVDB.h:2746
static KeyT CoordToKey(const CoordType &ijk)
Definition: NanoVDB.h:4186
static void read(std::istream &is, BufferT &buffer, Codec codec)
bool operator>(const Version &rhs) const
Definition: NanoVDB.h:968
const MaskType< LOG2DIM > & valueMask() const
Return a const reference to the bit mask of active voxels in this leaf node.
Definition: NanoVDB.h:6178
uint64_t mValueCount
Definition: NanoVDB.h:3334
Vec3T applyInverseMapF(const Vec3T &xyz) const
Apply the inverse affine mapping to a vector using 32bit floating point arithmetics.
Definition: NanoVDB.h:3257
ValueType operator*() const
Definition: NanoVDB.h:6132
const typename GridOrTreeOrRootT::RootNodeType type
Definition: NanoVDB.h:3463
TreeData & operator=(const TreeData &other)
Definition: NanoVDB.h:3947
DenseIterator beginDense()
Definition: NanoVDB.h:4577
bool isValueOn() const
Definition: NanoVDB.h:5139
void clear()
Reset this access to its initial state, i.e. with an empty cache.
Definition: NanoVDB.h:7248
bool operator<=(const Iterator &rhs) const
Definition: NanoVDB.h:2411
bool isIndex(GridType gridType)
Return true if the GridType maps to a special index type (not a POD integer type).
Definition: NanoVDB.h:855
typename ChildT::ValueType ValueT
Definition: NanoVDB.h:4860
bool isRootConnected() const
return true if RootData follows TreeData in memory without any extra padding
Definition: NanoVDB.h:3683
BBox< CoordT > mBBox
Definition: NanoVDB.h:4206
float Sqrt(float x)
Return the square root of a floating-point value.
Definition: NanoVDB.h:1233
AccessorType getAccessor() const
Definition: NanoVDB.h:4587
static Coord max()
Definition: NanoVDB.h:1339
Implements Tree::getDim(Coord)
Definition: NanoVDB.h:3477
Definition: NanoVDB.h:4352
uint32_t nodeCount(uint32_t level) const
Definition: NanoVDB.h:7581
const NodeTrait< RootT, 1 >::type * getFirstLower() const
Definition: NanoVDB.h:4140
Codec
Define compression codecs.
Definition: NanoVDB.h:7839
Vec3< T2 > operator*(T1 scalar, const Vec3< T2 > &vec)
Definition: NanoVDB.h:1693
int MinIndex(const Vec3T &v)
Definition: NanoVDB.h:1251
uint8_t mFlags
Definition: NanoVDB.h:5513
void setOff()
Definition: NanoVDB.h:2747
Internal nodes of a VDB treedim(),.
Definition: NanoVDB.h:4968
const Tile * tile(uint32_t n) const
Returns a non-const reference to the tile at the specified linear offset.
Definition: NanoVDB.h:4253
CoordT dim() const
Definition: NanoVDB.h:2462
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:6761
uint8_t & b()
Definition: NanoVDB.h:1951
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:5217
static constexpr uint32_t padding()
Return padding of this class in bytes, due to aliasing and 32B alignment.
Definition: NanoVDB.h:4892
#define NANOVDB_HOSTDEV_DISABLE_WARNING
Definition: NanoVDB.h:234
DenseIter(RootT *parent)
Definition: NanoVDB.h:4538
ValueT ValueType
Definition: NanoVDB.h:7190
BuildT TreeType
Definition: NanoVDB.h:3701
void setChild(const CoordType &k, const void *ptr, const RootData *data)
Definition: NanoVDB.h:4226
Base-class for quantized float leaf nodes.
Definition: NanoVDB.h:5504
uint64_t FloatType
Definition: NanoVDB.h:2031
bool probeValue(const Coord &ijk, typename remove_const< ChannelT >::type &v) const
return the state and updates the value of the specified voxel
Definition: NanoVDB.h:7796
static constexpr uint32_t padding()
Definition: NanoVDB.h:5755
uint64_t getValue(uint32_t i) const
Definition: NanoVDB.h:5886
FloatType average() const
Return a const reference to the average of all the active values encoded in this leaf node...
Definition: NanoVDB.h:6188
const typename GridOrTreeOrRootT::RootNodeType::ChildNodeType::ChildNodeType Type
Definition: NanoVDB.h:3433
Coord operator<<(IndexType n) const
Definition: NanoVDB.h:1368
Vec3T indexToWorld(const Vec3T &xyz) const
index to world space transformation
Definition: NanoVDB.h:3773
Vec4 & maxComponent(const Vec4 &other)
Perform a component-wise maximum with the other Coord.
Definition: NanoVDB.h:1825
Definition: NanoVDB.h:3940
bool hasAverage() const
Definition: NanoVDB.h:3840
ValueOnIterator cbeginValueOn() const
Definition: NanoVDB.h:6078
static constexpr uint64_t memUsage()
Definition: NanoVDB.h:5650
bool isCached2(const CoordType &ijk) const
Definition: NanoVDB.h:6974
Struct with all the member data of the InternalNode (useful during serialization of an openvdb Intern...
Definition: NanoVDB.h:4858
Vec3d voxelSize() const
Definition: NanoVDB.h:7577
Vec3T applyMapF(const Vec3T &xyz) const
Definition: NanoVDB.h:3599
Vec3 & operator=(const Vec3T< T2 > &rhs)
Definition: NanoVDB.h:1564
auto getNodeInfo(const CoordType &ijk) const
Definition: NanoVDB.h:6764
ValueOnIterator(const LeafNode *parent)
Definition: NanoVDB.h:6059
const typename GridT::TreeType Type
Definition: NanoVDB.h:3981
uint8_t & a()
Definition: NanoVDB.h:1952
float Clamp(float x, float a, float b)
Definition: NanoVDB.h:1131
bool isBitOn(uint8_t bit) const
Definition: NanoVDB.h:2787
BaseBBox & expand(const Vec3T &xyz)
Expand this bounding box to enclose point xyz.
Definition: NanoVDB.h:2250
CoordType getCoord() const
Definition: NanoVDB.h:5104
Dummy type for a 4bit quantization of float point values.
Definition: NanoVDB.h:282
ValueType getMin() const
Definition: NanoVDB.h:5999
bool probeValue(const CoordType &ijk, ValueType &v) const
return the state and updates the value of the specified voxel
Definition: NanoVDB.h:4636
typename ChildT::CoordType CoordT
Definition: NanoVDB.h:4863
float getDev() const
return the quantized standard deviation of the active values in this node
Definition: NanoVDB.h:5550
Return point to the lower internal node where Coord maps to one of its values, i.e. terminates.
Definition: NanoVDB.h:8134
Vec3T Vec3Type
Definition: NanoVDB.h:2304
void setRoot(const void *root)
Definition: NanoVDB.h:3953
uint64_t type
Definition: NanoVDB.h:682
static bool lessThan(const Coord &a, const Coord &b)
Definition: NanoVDB.h:1491
Vec3 & maxComponent(const Vec3 &other)
Perform a component-wise maximum with the other Coord.
Definition: NanoVDB.h:1647
const typename GridT::TreeType type
Definition: NanoVDB.h:3982
bool operator==(const Vec4 &rhs) const
Definition: NanoVDB.h:1755
Iterator operator++(int)
Definition: NanoVDB.h:2880
Vec3T worldToIndexDirF(const Vec3T &dir) const
transformation from world space direction to index space direction
Definition: NanoVDB.h:3806
Vec3< double > asVec3d() const
Return a double precision floating-point vector of this coordinate.
Definition: NanoVDB.h:1718
uint64_t getIndex(const Coord &ijk) const
Return the linear offset into a channel that maps to the specified coordinate.
Definition: NanoVDB.h:7787
static constexpr uint32_t padding()
Return padding of this class in bytes, due to aliasing and 32B alignment.
Definition: NanoVDB.h:5980
Mask & operator&=(const Mask &other)
Bitwise intersection.
Definition: NanoVDB.h:3069
CoordType getOrigin() const
Definition: NanoVDB.h:4371
uint8_t mFlags
Definition: NanoVDB.h:5750
const Vec3d & voxelSize() const
Return a const reference to the size of a voxel in world units.
Definition: NanoVDB.h:3762
static uint64_t memUsage()
Definition: NanoVDB.h:5804
void setValue(uint32_t offset, bool)
Definition: NanoVDB.h:5816
void toggle(uint32_t n)
Definition: NanoVDB.h:3066
static bool hasStats()
Definition: NanoVDB.h:5805
bool getMax() const
Definition: NanoVDB.h:5760
OffIterator beginOff() const
Definition: NanoVDB.h:2924
const ChildT * getChild(uint32_t n) const
Definition: NanoVDB.h:4919
uint64_t checksum() const
Return checksum of the grid buffer.
Definition: NanoVDB.h:3864
bool isPointData() const
Definition: NanoVDB.h:7561
static T value()
Definition: NanoVDB.h:1077
Definition: NanoVDB.h:505
static bool hasStats()
Definition: NanoVDB.h:5522
Coord floor() const
Round each component if this Vec<T> up to its integer value.
Definition: NanoVDB.h:1669
Struct with all the member data of the Grid (useful during serialization of an openvdb grid) ...
Definition: NanoVDB.h:3511
typename ChildT::template MaskType< LOG2 > MaskType
Definition: NanoVDB.h:4980
Implements Tree::isActive(Coord)
Definition: NanoVDB.h:3475
Coord & operator<<=(uint32_t n)
Definition: NanoVDB.h:1403
ValueIterator cbeginValueAll() const
Definition: NanoVDB.h:5074
Definition: NanoVDB.h:1966
Iterator begin() const
Definition: NanoVDB.h:2420
void setAvg(float avg)
Definition: NanoVDB.h:5559
FloatType stdDeviation() const
Return a const reference to the standard deviation of all the active values encoded in this leaf node...
Definition: NanoVDB.h:6194
void setOn(uint32_t offset)
Definition: NanoVDB.h:5472
NodeT * operator->() const
Definition: NanoVDB.h:5018
void init(std::initializer_list< GridFlags > list={GridFlags::IsBreadthFirst}, uint64_t gridSize=0u, const Map &map=Map(), GridType gridType=GridType::Unknown, GridClass gridClass=GridClass::Unknown)
Definition: NanoVDB.h:3538
void setOn(uint32_t offset)
Definition: NanoVDB.h:5537
const uint32_t & getTableSize() const
Definition: NanoVDB.h:4604
uint32_t mData0
Definition: NanoVDB.h:3529
static uint32_t bitCount()
Return the number of bits available in this Mask.
Definition: NanoVDB.h:2834
bool operator==(const Version &rhs) const
Definition: NanoVDB.h:965
type for indexing points into voxels
Definition: NanoVDB.h:302
void setMax(const ValueType &v)
Definition: NanoVDB.h:5480
uint8_t & operator[](int n)
Definition: NanoVDB.h:1942
ValueOnIterator cbeginValueOn() const
Definition: NanoVDB.h:5108
typename ChildT::FloatType StatsT
Definition: NanoVDB.h:4862
Definition: NanoVDB.h:2685
const Vec3T & operator[](int i) const
Definition: NanoVDB.h:2237
const uint8_t & a() const
Definition: NanoVDB.h:1948
DataT * mData
Definition: NanoVDB.h:4357
static int64_t PtrDiff(const T1 *p, const T2 *q)
Compute the distance, in bytes, between two pointers.
Definition: NanoVDB.h:780
bool Type
Definition: NanoVDB.h:688
Vec3 operator*(const T &s) const
Definition: NanoVDB.h:1595
bool isMaskOn(std::initializer_list< MaskT > list) const
return true if any of the masks in the list are on
Definition: NanoVDB.h:2795
void setDev(const FloatType &)
Definition: NanoVDB.h:6007
static uint64_t memUsage()
Definition: NanoVDB.h:5520
T & operator[](int i)
Definition: NanoVDB.h:1769
Definition: NanoVDB.h:2892
ValueOnIterator beginValueOn() const
Definition: NanoVDB.h:5107
uint64_t gridPoints(const AttT *&begin, const AttT *&end) const
Return the total number of point in the grid and set the iterators to the complete range of points...
Definition: NanoVDB.h:7616
uint32_t rootTableSize() const
return the root table has size
Definition: NanoVDB.h:3670
StatsT mAverage
Definition: NanoVDB.h:4885
Tolerance for floating-point comparison.
Definition: NanoVDB.h:1019
Definition: NanoVDB.h:4528
void setFirstNode(const NodeT *node)
Definition: NanoVDB.h:3958
CoordT mBBoxMin
Definition: NanoVDB.h:5511
bool operator!=(const Coord &rhs) const
Definition: NanoVDB.h:1395
typename ChildT::CoordType CoordT
Definition: NanoVDB.h:4178
T Pow2(T x)
Definition: NanoVDB.h:1168
static uint32_t voxelCount()
Return the total number of voxels (e.g. values) encoded in this leaf node.
Definition: NanoVDB.h:6237
void setAvg(const StatsT &v)
Definition: NanoVDB.h:4315
static Coord min()
Definition: NanoVDB.h:1341
MaskT mValueMask
Definition: NanoVDB.h:4880
Vec3T dim() const
Definition: NanoVDB.h:2339
uint16_t mMin
Definition: NanoVDB.h:5518
typename ChildT::FloatType StatsT
Definition: NanoVDB.h:4179
typename GridOrTreeOrRootT::RootNodeType::ChildNodeType Type
Definition: NanoVDB.h:3440
static uint64_t memUsage()
Definition: NanoVDB.h:5938
LeafNodeType * getFirstLeaf()
Template specializations of getFirstNode.
Definition: NanoVDB.h:4137
typename FloatTraits< ValueType >::FloatType FloatType
Definition: NanoVDB.h:5963
Coord offsetBy(ValueType n) const
Definition: NanoVDB.h:1487
bool probeValue(const CoordType &ijk, ValueType &v) const
Definition: NanoVDB.h:6992
Vec3T worldToIndex(const Vec3T &xyz) const
world to index space transformation
Definition: NanoVDB.h:3769
CoordType origin() const
Return the origin in index space of this leaf node.
Definition: NanoVDB.h:5180
ConstChildIterator cbeginChild() const
Definition: NanoVDB.h:4432
Like ValueOnIndex but with a mutable mask.
Definition: NanoVDB.h:267
size_t memUsage() const
Definition: NanoVDB.h:5687
GridMetaData(const GridData *gridData)
Definition: NanoVDB.h:7526
Vec4 operator-(const Vec4 &v) const
Definition: NanoVDB.h:1781
ReadAccessor(const GridT &grid)
Constructor from a grid.
Definition: NanoVDB.h:6724
const typename GridOrTreeOrRootT::LeafNodeType Type
Definition: NanoVDB.h:3418
void setMin(const ValueType &)
Definition: NanoVDB.h:5818
MaskT< LOG2DIM > mValues
Definition: NanoVDB.h:5752
uint64_t getMin() const
Definition: NanoVDB.h:5908
float getValue(uint32_t i) const
Definition: NanoVDB.h:5627
This is a convenient class that allows for access to grid meta-data that are independent of the value...
Definition: NanoVDB.h:7510
static bool isValid(const void *p)
return true if the specified pointer is aligned and not NULL
Definition: NanoVDB.h:743
uint8_t * nodePtr()
Return a non-const uint8_t pointer to the first node at LEVEL.
Definition: NanoVDB.h:3633
ValueType getLastValue() const
Return the last value in this leaf node.
Definition: NanoVDB.h:6259
const std::enable_if<!VecTraits< T >::IsVec, T >::type & max(const T &a, const T &b)
Definition: Composite.h:110
uint32_t countOn() const
Return the total number of set bits in this Mask.
Definition: NanoVDB.h:2840
const uint8_t * treePtr() const
Definition: NanoVDB.h:3614
void setMinMaxOn(bool on=true)
Definition: NanoVDB.h:3572
bool isMaskOff(MaskT mask) const
Definition: NanoVDB.h:2792
void setValue(const CoordType &k, bool s, const ValueType &v)
Definition: NanoVDB.h:4233
uint32_t mTableSize
Definition: NanoVDB.h:4207
typename BuildT::BuildType BuildType
Definition: NanoVDB.h:3709
CoordT origin() const
Return the origin in index space of this leaf node.
Definition: NanoVDB.h:6199
typename T::ValueType ElementType
Definition: NanoVDB.h:1986
Iterator operator++(int)
Definition: NanoVDB.h:2390
DenseIterator & operator++()
Definition: NanoVDB.h:2903
Definition: NanoVDB.h:6029
typename RootT::CoordType CoordType
Definition: NanoVDB.h:4005
float type
Definition: NanoVDB.h:724
defines a tree type from a grid type while preserving constness
Definition: NanoVDB.h:3973
ValueType getFirstValue() const
If the first entry in this node&#39;s table is a tile, return the tile&#39;s value. Otherwise, return the result of calling getFirstValue() on the child.
Definition: NanoVDB.h:5202
typename match_const< Tile, RootT >::type TileT
Definition: NanoVDB.h:4356
bool probeValue(const CoordType &ijk, ValueType &v) const
Definition: NanoVDB.h:6613
bool isFogVolume() const
Definition: NanoVDB.h:7557
void setOff(uint32_t n)
Set the specified bit off.
Definition: NanoVDB.h:3009
ReadAccessor(const TreeT &tree)
Constructor from a tree.
Definition: NanoVDB.h:6918
uint32_t getMinor() const
Definition: NanoVDB.h:972
ChannelT & operator()(const Coord &ijk) const
Definition: NanoVDB.h:7792
void localToGlobalCoord(Coord &ijk) const
Converts (in place) a local index coordinate to a global index coordinate.
Definition: NanoVDB.h:6212
bool isInside(const BBox &b) const
Return true if the given bounding box is inside this bounding box.
Definition: NanoVDB.h:2470
#define __hostdev__
Definition: NanoVDB.h:213
void setDev(const FloatType &)
Definition: NanoVDB.h:5821
const uint64_t * words() const
Definition: NanoVDB.h:2950
ValueIterator beginValue() const
Definition: NanoVDB.h:6161
Visits child nodes of this node only.
Definition: NanoVDB.h:4994
ChildIter & operator++()
Definition: NanoVDB.h:4412
bool safeCast() const
return true if the RootData follows right after the TreeData. If so, this implies that it&#39;s safe to c...
Definition: NanoVDB.h:7541
ReadAccessor(const GridT &grid)
Constructor from a grid.
Definition: NanoVDB.h:6582
FloatType getAvg() const
Definition: NanoVDB.h:5476
const Tile * probeTile(const CoordT &ijk) const
Definition: NanoVDB.h:4289
enable_if< is_same< T, Point >::value, const uint64_t & >::type pointCount() const
Return the total number of points indexed by this PointGrid.
Definition: NanoVDB.h:3750
const ChildNodeType * probeChild(const CoordType &ijk) const
Definition: NanoVDB.h:5255
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:4633
void setValue(uint32_t offset, uint16_t value)
Definition: NanoVDB.h:5992
Iterator & operator++()
Definition: NanoVDB.h:2376
uint64_t Type
Definition: NanoVDB.h:681
uint8_t ValueType
Definition: NanoVDB.h:1869
ValueOffIterator(const LeafNode *parent)
Definition: NanoVDB.h:6092
typename Mask< 3 >::template Iterator< ON > MaskIterT
Definition: NanoVDB.h:6045
GridType mDataType
Definition: NanoVDB.h:3338
Leaf nodes of the VDB tree. (defaults to 8x8x8 = 512 voxels)
Definition: NanoVDB.h:6026
uint32_t & packed()
Definition: NanoVDB.h:1944
static size_t memUsage()
Return the memory footprint in bytes of this Mask.
Definition: NanoVDB.h:2831
Vec3 operator-() const
Definition: NanoVDB.h:1588
void setDev(const StatsT &v)
Definition: NanoVDB.h:4954
Vec3T applyInverseJacobianF(const Vec3T &xyz) const
Apply the linear inverse 3x3 transformation to an input 3d vector using 32bit floating point arithmet...
Definition: NanoVDB.h:3278
BBox< Vec3d > transform(const Map &map) const
transform this coordinate bounding box by the specified map
Definition: NanoVDB.h:2499
T Pow3(T x)
Definition: NanoVDB.h:1174
Iterator(const BBox &b)
Definition: NanoVDB.h:2366
const char * shortGridName() const
Return a c-string with the name of this grid, truncated to 255 characters.
Definition: NanoVDB.h:3861
ChildT * getChild(uint32_t n)
Returns a pointer to the child node at the specifed linear offset.
Definition: NanoVDB.h:4914
void setOrigin(const T &ijk)
Definition: NanoVDB.h:5824
BuildT BuildType
Definition: NanoVDB.h:6890
bool FloatType
Definition: NanoVDB.h:5744
static bool hasStats()
Definition: NanoVDB.h:5757
double mTaperD
Definition: NanoVDB.h:3166
void setDev(const FloatType &)
Definition: NanoVDB.h:5862
uint32_t dim
Definition: NanoVDB.h:8197
const DataType * data() const
Definition: NanoVDB.h:4591
void setMax(const ValueT &v)
Definition: NanoVDB.h:4314
uint64_t first(uint32_t i) const
Definition: NanoVDB.h:5988
MaskT mChildMask
Definition: NanoVDB.h:4881
bool isActive(const CoordType &ijk) const
Definition: NanoVDB.h:6765
ChildT * getChild(const Tile *tile)
Returns a const reference to the child node in the specified tile.
Definition: NanoVDB.h:4297
math::Extrema extrema(const IterT &iter, bool threaded=true)
Iterate over a scalar grid and compute extrema (min/max) of the values of the voxels that are visited...
Definition: Statistics.h:354
bool isCached(const CoordType &ijk) const
Definition: NanoVDB.h:6749
CoordBBox bbox() const
Return the index bounding box of all the active values in this tree, i.e. in all nodes of the tree...
Definition: NanoVDB.h:3963
ValueOnIterator beginValueOn() const
Definition: NanoVDB.h:6077
Signed (i, j, k) 32-bit integer coordinate class, similar to openvdb::math::Coord.
Definition: NanoVDB.h:1301
Vec3T matMultT(const float *mat, const Vec3T &xyz)
Multiply the transposed of a 3x3 matrix and a 3d vector using 32bit floating point arithmetics...
Definition: NanoVDB.h:2191
const DataType * data() const
Definition: NanoVDB.h:4021
bool hasLongGridName() const
Definition: NanoVDB.h:7566
int32_t y() const
Definition: NanoVDB.h:1332
ValueIterator()
Definition: NanoVDB.h:6120
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5446
ChildNodeType * probeChild(const CoordType &ijk)
Definition: NanoVDB.h:4690
BaseIter(DataT *data=nullptr, uint32_t n=0)
Definition: NanoVDB.h:4359
Definition: NanoVDB.h:518
C++11 implementation of std::is_floating_point.
Definition: NanoVDB.h:462
ValueOnIter operator++(int)
Definition: NanoVDB.h:4513
typename GridOrTreeOrRootT::RootNodeType Type
Definition: NanoVDB.h:3454
bool isBreadthFirst() const
Definition: NanoVDB.h:7569
void setMin(const ValueT &v)
Definition: NanoVDB.h:4951
Definition: NanoVDB.h:4223
uint64_t checksum() const
Definition: NanoVDB.h:7582
FloatType mAverage
Definition: NanoVDB.h:5450
bool isValue() const
Definition: NanoVDB.h:4241
BuildT BuildType
Definition: NanoVDB.h:6557
Vec3T applyInverseJacobian(const Vec3T &xyz) const
Apply the linear inverse 3x3 transformation to an input 3d vector using 64bit floating point arithmet...
Definition: NanoVDB.h:3269
ValueT ValueType
Definition: NanoVDB.h:6891
const std::enable_if<!VecTraits< T >::IsVec, T >::type & min(const T &a, const T &b)
Definition: Composite.h:106
static constexpr int64_t memUsage()
Definition: NanoVDB.h:5619
Definition: NanoVDB.h:2232
BlindDataT * getBlindData(uint32_t n)
Definition: NanoVDB.h:3897
ChannelT & getValue(const Coord &ijk) const
Return the value from a cached channel that maps to the specified coordinate.
Definition: NanoVDB.h:7791
AccessorType getAccessor() const
Definition: NanoVDB.h:4040
bool isActive() const
Definition: NanoVDB.h:4242
float Type
Definition: NanoVDB.h:695
typename UpperNodeType::ChildNodeType LowerNodeType
Definition: NanoVDB.h:4337
StatsT mStdDevi
Definition: NanoVDB.h:4213
Vec3 & operator+=(const Vec3 &v)
Definition: NanoVDB.h:1597
void setMaskOff(MaskT mask)
Definition: NanoVDB.h:2766
Coord(ValueType i, ValueType j, ValueType k)
Initializes coordinate to the given signed integers.
Definition: NanoVDB.h:1321
bool hasLongGridName() const
Definition: NanoVDB.h:3839
ReadAccessor(const RootT &root)
Constructor from a root node.
Definition: NanoVDB.h:6716
const uint32_t & tileCount() const
Return the number of tiles encoded in this root node.
Definition: NanoVDB.h:4603
C++11 implementation of std::enable_if.
Definition: NanoVDB.h:492
ValueT value
Definition: NanoVDB.h:4869
uint64_t gridSize() const
Return memory usage in bytes for this class only.
Definition: NanoVDB.h:3730
Vec4(const Vec4T< T2 > &v)
Definition: NanoVDB.h:1750
CoordBBox indexBBox
Definition: NanoVDB.h:7879
const LeafNode * probeLeaf(const CoordT &) const
Definition: NanoVDB.h:6294
ValueType getValue(int i, int j, int k) const
Definition: NanoVDB.h:6608
C++11 implementation of std::is_same.
Definition: NanoVDB.h:441
bool hasBBox() const
Definition: NanoVDB.h:7565
typename GridOrTreeOrRootT::RootNodeType type
Definition: NanoVDB.h:3455
static uint64_t memUsage()
Definition: NanoVDB.h:5756
Mask & operator|=(const Mask &other)
Bitwise union.
Definition: NanoVDB.h:3078
ValueType getValue(const CoordType &ijk) const
Definition: NanoVDB.h:6604
static uint32_t dim()
Return the dimension, in voxel units, of this internal node (typically 8*16 or 8*16*32) ...
Definition: NanoVDB.h:5166
uint32_t gridCount() const
Return total number of grids in the buffer.
Definition: NanoVDB.h:3736
enable_if<!is_same< MaskT, Mask >::value, Mask & >::type operator=(const MaskT &other)
Assignment operator that works with openvdb::util::NodeMask.
Definition: NanoVDB.h:2954
bool getAvg() const
Definition: NanoVDB.h:5761
uint64_t getDev() const
Definition: NanoVDB.h:5885
ConstChildIterator cbeginChild() const
Definition: NanoVDB.h:5035
ChildT ChildNodeType
Definition: NanoVDB.h:4976
Coord & minComponent(const Coord &other)
Perform a component-wise minimum with the other Coord.
Definition: NanoVDB.h:1443
typename DataType::BuildT BuildType
Definition: NanoVDB.h:4974
float getValue(uint32_t i) const
Definition: NanoVDB.h:5591
uint32_t mSize
Definition: NanoVDB.h:4358
ValueIterator operator++(int)
Definition: NanoVDB.h:6153
typename DataType::ValueType ValueType
Definition: NanoVDB.h:6037
float type
Definition: NanoVDB.h:717
const CoordBBox & indexBBox() const
return AABB of active values in index space
Definition: NanoVDB.h:3667
bool operator==(const Rgba8 &rhs) const
Definition: NanoVDB.h:1931
Coord(ValueType *ptr)
Definition: NanoVDB.h:1326
uint64_t type
Definition: NanoVDB.h:731
typename ChildT::LeafNodeType LeafNodeType
Definition: NanoVDB.h:4975
Mask(const Mask &other)
Copy constructor.
Definition: NanoVDB.h:2942
const uint64_t & firstOffset() const
Definition: NanoVDB.h:5858
bool isActive(uint32_t n) const
Definition: NanoVDB.h:6274
const ValueType & background() const
Return a const reference to the background value.
Definition: NanoVDB.h:4056
Coord & operator=(const CoordT &other)
Assignment operator that works with openvdb::Coord.
Definition: NanoVDB.h:1355
uint64_t Type
Definition: NanoVDB.h:674
float getAvg() const
return the quantized average of the active values in this node
Definition: NanoVDB.h:5546
static void * memcpy64(void *dst, const void *src, size_t word_count)
copy 64 bit words from src to dst
Definition: NanoVDB.h:871
ValueOffIterator()
Definition: NanoVDB.h:6087
bool isActive() const
Definition: NanoVDB.h:4456
GridBlindDataClass
Blind-data Classes that are currently supported by NanoVDB.
Definition: NanoVDB.h:416
MaskT< LOG2DIM > mValueMask
Definition: NanoVDB.h:5970
bool isGridIndex() const
Definition: NanoVDB.h:7560
DenseIterator operator++(int)
Definition: NanoVDB.h:2908
Vec4(T x)
Definition: NanoVDB.h:1736
const typename GridOrTreeOrRootT::RootNodeType Type
Definition: NanoVDB.h:3462
Visits all active values in a leaf node.
Definition: NanoVDB.h:6048
int blindDataCount() const
Definition: NanoVDB.h:7578
static uint32_t dim()
Return the dimension, in index space, of this leaf node (typically 8 as for openvdb leaf nodes!) ...
Definition: NanoVDB.h:6220
ChannelT * setChannel(uint32_t channelID)
Change to an internal channel, assuming it exists as as blind data in the IndexGrid.
Definition: NanoVDB.h:7781
Coord operator-() const
Definition: NanoVDB.h:1426