Nilorea Library
C utilities for networking, threading, graphics
Loading...
Searching...
No Matches
AVRO: binary format encoding/decoding with JSON conversion

Data Structures

struct  AVRO_FIELD
 Avro schema field (for records) More...
 
struct  AVRO_READER
 Avro read cursor for decoding. More...
 
struct  AVRO_SCHEMA
 Avro schema definition. More...
 

Macros

#define AVRO_MAGIC   "Obj\x01"
 Avro object container file magic bytes.
 
#define AVRO_MAGIC_LEN   4
 Avro magic length.
 
#define AVRO_SYNC_LEN   16
 Avro sync marker length.
 

Enumerations

enum  AVRO_TYPE {
  AVRO_NULL = 0 , AVRO_BOOLEAN , AVRO_INT , AVRO_LONG ,
  AVRO_FLOAT , AVRO_DOUBLE , AVRO_BYTES , AVRO_STRING ,
  AVRO_RECORD , AVRO_ENUM , AVRO_ARRAY , AVRO_MAP ,
  AVRO_UNION , AVRO_FIXED
}
 Avro schema type enumeration. More...
 

Functions

cJSON * avro_decode_container (const AVRO_SCHEMA *schema, const N_STR *avro_data)
 Decode an Avro container format N_STR into a cJSON array of records.
 
cJSON * avro_decode_datum (AVRO_READER *reader, const AVRO_SCHEMA *schema)
 Decode an Avro binary datum into cJSON according to schema.
 
int avro_decode_long (AVRO_READER *reader, int64_t *value)
 Decode a zig-zag varint from reader into a 64-bit signed integer.
 
N_STRavro_encode_container (const AVRO_SCHEMA *schema, const cJSON *records)
 Encode a cJSON array of records into Avro container format N_STR.
 
int avro_encode_datum (N_STR **dest, const AVRO_SCHEMA *schema, const cJSON *json)
 Encode a cJSON value as Avro binary according to schema.
 
int avro_encode_long (N_STR **dest, int64_t value)
 Encode a 64-bit signed integer as zig-zag varint into N_STR.
 
int avro_file_to_json (const char *avro_filename, const char *schema_filename, const char *json_filename)
 Read an Avro object container file and produce a JSON file using a schema file.
 
int avro_json_to_file (const char *avro_filename, const char *schema_filename, const char *json_filename)
 Write an Avro object container file from a JSON file and schema file.
 
N_STRavro_nstr_avro_to_json (const N_STR *schema_nstr, const N_STR *avro_nstr)
 Convert Avro N_STR to JSON N_STR using schema N_STR (all in-memory)
 
N_STRavro_nstr_json_to_avro (const N_STR *schema_nstr, const N_STR *json_nstr)
 Convert JSON N_STR to Avro N_STR using schema N_STR (all in-memory)
 
void avro_schema_free (AVRO_SCHEMA **schema)
 Free an Avro schema.
 
AVRO_SCHEMAavro_schema_from_cjson (const cJSON *json)
 Parse an Avro schema from a cJSON object.
 
AVRO_SCHEMAavro_schema_parse (const char *json_str)
 Parse an Avro schema from a JSON string.
 
AVRO_SCHEMAavro_schema_parse_nstr (const N_STR *schema_nstr)
 Parse schema from N_STR.
 
char * avro_schema_to_json (const AVRO_SCHEMA *schema)
 Convert an Avro schema back to JSON string (caller must free)
 

Detailed Description


Data Structure Documentation

◆ AVRO_FIELD

struct AVRO_FIELD

Avro schema field (for records)

Definition at line 76 of file n_avro.h.

+ Collaboration diagram for AVRO_FIELD:
Data Fields
char * name field name
AVRO_SCHEMA * schema field schema

◆ AVRO_READER

struct AVRO_READER

Avro read cursor for decoding.

Definition at line 112 of file n_avro.h.

+ Collaboration diagram for AVRO_READER:
Data Fields
const unsigned char * data data buffer
size_t pos current position
size_t size total size

◆ AVRO_SCHEMA

struct AVRO_SCHEMA

Avro schema definition.

Forward declaration.

Definition at line 84 of file n_avro.h.

+ Collaboration diagram for AVRO_SCHEMA:
Data Fields
AVRO_FIELD * fields namespace (for record, enum)

fields array (for record)

size_t fixed_size fixed size
AVRO_SCHEMA * items item schema (for array)
char * name name (for record, enum, fixed)
size_t nb_branches number of union branches
size_t nb_fields number of fields (for record)
size_t nb_symbols number of enum symbols
char ** symbols enum symbols
AVRO_TYPE type schema type
AVRO_SCHEMA ** union_branches union branch schemas
AVRO_SCHEMA * values value schema (for map)

Macro Definition Documentation

◆ AVRO_MAGIC

#define AVRO_MAGIC   "Obj\x01"

Avro object container file magic bytes.

Definition at line 48 of file n_avro.h.

◆ AVRO_MAGIC_LEN

#define AVRO_MAGIC_LEN   4

Avro magic length.

Definition at line 50 of file n_avro.h.

◆ AVRO_SYNC_LEN

#define AVRO_SYNC_LEN   16

Avro sync marker length.

Definition at line 52 of file n_avro.h.

Enumeration Type Documentation

◆ AVRO_TYPE

enum AVRO_TYPE

Avro schema type enumeration.

Enumerator
AVRO_NULL 
AVRO_BOOLEAN 
AVRO_INT 
AVRO_LONG 
AVRO_FLOAT 
AVRO_DOUBLE 
AVRO_BYTES 
AVRO_STRING 
AVRO_RECORD 
AVRO_ENUM 
AVRO_ARRAY 
AVRO_MAP 
AVRO_UNION 
AVRO_FIXED 

Definition at line 55 of file n_avro.h.

Function Documentation

◆ avro_decode_container()

cJSON * avro_decode_container ( const AVRO_SCHEMA schema,
const N_STR avro_data 
)

Decode an Avro container format N_STR into a cJSON array of records.

Definition at line 1025 of file n_avro.c.

References __n_assert, avro_decode_bytes_raw(), avro_decode_datum(), avro_decode_long(), AVRO_MAGIC, AVRO_MAGIC_LEN, AVRO_SYNC_LEN, AVRO_READER::data, N_STR::data, Free, key, LOG_ERR, LOG_WARNING, n_log, AVRO_READER::pos, AVRO_READER::size, and N_STR::written.

Referenced by avro_file_to_json(), and avro_nstr_avro_to_json().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_decode_datum()

◆ avro_decode_long()

int avro_decode_long ( AVRO_READER reader,
int64_t *  value 
)

Decode a zig-zag varint from reader into a 64-bit signed integer.

Definition at line 394 of file n_avro.c.

References __n_assert, AVRO_READER::data, LOG_ERR, n_log, AVRO_READER::pos, and AVRO_READER::size.

Referenced by avro_decode_bytes_raw(), avro_decode_container(), and avro_decode_datum().

+ Here is the caller graph for this function:

◆ avro_encode_container()

N_STR * avro_encode_container ( const AVRO_SCHEMA schema,
const cJSON *  records 
)

Encode a cJSON array of records into Avro container format N_STR.

Definition at line 942 of file n_avro.c.

References __n_assert, avro_encode_datum(), avro_encode_long(), AVRO_MAGIC, AVRO_MAGIC_LEN, avro_schema_to_json(), AVRO_SYNC_LEN, N_STR::data, Free, free_nstr, LOG_ERR, n_log, nstrcat_ex(), nstrprintf, and N_STR::written.

Referenced by avro_json_to_file(), and avro_nstr_json_to_avro().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_encode_datum()

◆ avro_encode_long()

int avro_encode_long ( N_STR **  dest,
int64_t  value 
)

Encode a 64-bit signed integer as zig-zag varint into N_STR.

Definition at line 376 of file n_avro.c.

References __n_assert, and nstrcat_ex().

Referenced by avro_encode_bytes_raw(), avro_encode_container(), and avro_encode_datum().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_file_to_json()

int avro_file_to_json ( const char *  avro_filename,
const char *  schema_filename,
const char *  json_filename 
)

Read an Avro object container file and produce a JSON file using a schema file.

Definition at line 1210 of file n_avro.c.

References __n_assert, avro_decode_container(), avro_schema_free(), avro_schema_parse(), char_to_nstr(), N_STR::data, file_to_nstr(), free_nstr, LOG_ERR, LOG_INFO, n_log, and nstr_to_file().

Referenced by main().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_json_to_file()

int avro_json_to_file ( const char *  avro_filename,
const char *  schema_filename,
const char *  json_filename 
)

Write an Avro object container file from a JSON file and schema file.

Definition at line 1146 of file n_avro.c.

References __n_assert, _str, avro_encode_container(), avro_schema_free(), avro_schema_parse(), N_STR::data, file_to_nstr(), free_nstr, LOG_ERR, LOG_INFO, n_log, and nstr_to_file().

Referenced by main().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_nstr_avro_to_json()

N_STR * avro_nstr_avro_to_json ( const N_STR schema_nstr,
const N_STR avro_nstr 
)

Convert Avro N_STR to JSON N_STR using schema N_STR (all in-memory)

Definition at line 1315 of file n_avro.c.

References __n_assert, avro_decode_container(), avro_schema_free(), avro_schema_parse(), char_to_nstr(), N_STR::data, LOG_ERR, and n_log.

Referenced by main().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_nstr_json_to_avro()

N_STR * avro_nstr_json_to_avro ( const N_STR schema_nstr,
const N_STR json_nstr 
)

Convert JSON N_STR to Avro N_STR using schema N_STR (all in-memory)

Definition at line 1279 of file n_avro.c.

References __n_assert, _str, avro_encode_container(), avro_schema_free(), avro_schema_parse(), N_STR::data, LOG_ERR, and n_log.

Referenced by main().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_schema_free()

◆ avro_schema_from_cjson()

◆ avro_schema_parse()

AVRO_SCHEMA * avro_schema_parse ( const char *  json_str)

Parse an Avro schema from a JSON string.

Definition at line 211 of file n_avro.c.

References __n_assert, and avro_schema_from_cjson().

Referenced by avro_file_to_json(), avro_json_to_file(), avro_nstr_avro_to_json(), avro_nstr_json_to_avro(), and avro_schema_parse_nstr().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_schema_parse_nstr()

AVRO_SCHEMA * avro_schema_parse_nstr ( const N_STR schema_nstr)

Parse schema from N_STR.

Definition at line 1272 of file n_avro.c.

References __n_assert, avro_schema_parse(), and N_STR::data.

Referenced by main().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ avro_schema_to_json()