Module memoryblob

Auxiliary library for creating memory blobs, for instance to recycle or share native resources.

Some concurrency primitives are provided, built upon libcuckoo. (At the moment, Corona uses libstdc++ on iOS, which makes libcuckoo support too difficult. A basic mutex / map combination is used instead, but alternatives are under investigation.)

To use the plugin, add the following in build.settings:

plugins = {
   ["plugin.MemoryBlob"] = { publisherId = "com.xibalbastudios" }
 }

Sample code is available here.

Once this plugin has been loaded, native plugins may create and manage blobs via the BlobXS API. Until then, any calls to the API will be suitable no-ops. (Documentation is lacking at the moment!)

Blobs implement the ByteReader protocol.

The Bytes type—specified in a few of the blob methods—may be any object that implements ByteReader, including strings.

Functions and sections that begin with (WIP) describe work in progress. These features are not considered stable, but give a reasonable idea of what to expect.


From libcuckoo's license:

Copyright (C) 2013, Carnegie Mellon University and Intel Corporation

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Apache License 2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.


From the license of CityHash (used by libcuckoo):

Copyright (c) 2011 Google, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


Functions

ExistsInStorage (id) (WIP) Check whether an entry exists in storage.
GetBlobDispatcher () Get the dispatcher used by blob-related events, in order to add / remove listeners.
IsBlob (object, type) Indicates whether an object is a MemoryBlob.
New (opts) Create a new memory blob.
MemoryBlob:Append (bytes) Add bytes to the end of the blob.
MemoryBlob:Clone () Create a clone of the blob.
MemoryBlob:GetBytes ([i1=1[, i2=#self]]) Dump the blob's contents as a string of bytes.
MemoryBlob:GetProperties (out)

Populate a table with the following blob properties:

  • alignment: As per New, an integer ≥ 0, with 0 meaning default alignment.
MemoryBlob:Insert (pos, bytes) Insert bytes into the blob, starting at position pos.
MemoryBlob:IsLocked () Indicates whether writes to this blob are restricted (via the native API).
MemoryBlob:__len () Metamethod.
MemoryBlob:Remove ([i1=1[, i2=#self]]) Remove a range of bytes from the blob.
MemoryBlob:Submit () (WIP) Submit this blob's contents to storage.
MemoryBlob:Sync (id) (WIP) Synchronize this blob with an entry in storage, which then ceases to exist.
MemoryBlob:Write (pos, bytes) Write bytes to the blob, starting at position pos, overwriting the blob's contents.


Functions

ExistsInStorage (id)
(WIP) Check whether an entry exists in storage.

Parameters:

Returns:

    boolean The entry exists?
GetBlobDispatcher ()
Get the dispatcher used by blob-related events, in order to add / remove listeners.

Calling this outside Corona's main thread results in an error.

Returns:

    EventDispatcher Dispatcher.

See also:

IsBlob (object, type)
Indicates whether an object is a MemoryBlob.

Parameters:

  • object Object to check.
  • type optional string If present, refine the query to only consider blobs of a given type, cf. New.

Returns:

    boolean Is object a blob?
New (opts)
Create a new memory blob.

No assumptions should be made about the original contents; whatever the allocator provides is left intact.

Parameters:

  • opts table, uint or nil Blob creation options.

    When opts is a table, it may contain the following options:

    • alignment: If specified, the memory alignment, which must be a multiple of 2, ≥ 4. The blob's memory will start at an address that is a multiple of this value, which is useful and / or needed e.g. for SIMD operations. By default, blobs use the Lua allocator's alignment.

      Currently, the upper limit is 1024, one level beyond AVX2 support.
    • resizable: If true, the blob can be resized. Off by default.
    • size: Blob size in bytes, ≥ 0. For resizable blobs, this is the blob's initial size; otherwise, it specifies the fixed size. If absent, 0.
    • type: A string that will be used to name the blob userdata's metatable; if absent, uses a default.

      Blobs themselves make no further use of this value; rather it is exposed as a convenience, e.g. so plugin authors can identify their own blobs via IsBlob.

    If opts is an integer, it specifies the fixed size (≥ 0) of the blob to create.

    For any other value of opts, a resizable blob will be created.

Returns:

    MemoryBlob The new blob.
MemoryBlob:Append (bytes)
Add bytes to the end of the blob.

This is a no-op for fixed-size and locked blobs.

Parameters:

  • bytes Bytes Bytes to write.

Returns:

    uint Number of bytes actually written.

See also:

MemoryBlob:Clone ()
Create a clone of the blob.

The clone inherits the alignment, resizability, size, and type of the original blob.

Initially, the clone will contain a copy of the parent's current contents. The two blobs are independent, however; changes to one's contents are not reflected in the other.

Clones begin unlocked.

Returns:

    MemoryBlob Cloned blob.
MemoryBlob:GetBytes ([i1=1[, i2=#self]])
Dump the blob's contents as a string of bytes.

If either index refers to a position outside the blob, or i1 > i2 (after normalizing any negative indices), an empty string is returned.

Parameters:

  • i1 uint Index of first byte. If negative, the index counts down from the end of the blob, much like certain Lua string functions. (default 1)
  • i2 uint Index of last byte. Again, indices may be negative. (default #self)

Returns:

    string Copy of current blob contents.

See also:

MemoryBlob:GetProperties (out)

Populate a table with the following blob properties:

  • alignment: As per New, an integer ≥ 0, with 0 meaning default alignment.
  • resizable: As per New, a boolean indicating resizability.

Parameters:

  • out optional table Table to populate and return. If absent, a fresh table is created.

Returns:

    table Properties table.
MemoryBlob:Insert (pos, bytes)
Insert bytes into the blob, starting at position pos.

The current contents from pos onward are moved ahead #bytes positions to make room. In the case of fixed-size blobs, any bytes moved beyond the end of the blob are thrown away.

This is a no-op for locked blobs.

Parameters:

  • pos int Insert position, between 1 and #self, inclusive. A negative index may also be provided, cf. MemoryBlob:GetBytes for details.

    Resizable blobs may also use position #self + 1, with behavior like self:Append(bytes).

    No bytes are inserted when the position lies outside the legal range.

  • bytes Bytes Bytes to insert.

Returns:

    uint Number of bytes actually inserted.

See also:

MemoryBlob:IsLocked ()
Indicates whether writes to this blob are restricted (via the native API).

Certain MemoryBlob methods will early-out when given a locked blob, cf. various summaries.

Returns:

    boolean The blob is locked?
MemoryBlob:__len ()
Metamethod.

Returns:

    uint Current size of blob, in bytes.
MemoryBlob:Remove ([i1=1[, i2=#self]])
Remove a range of bytes from the blob.

Any bytes beyond i2 will be moved down to fill the vacated positions.

Naturally, resizable blobs will shrink. Since fixed-size blobs cannot do this, there will be i2 - i1 + 1 "extra" bytes at the end, after elements move down; these will be left as-is.

This is a no-op for locked blobs.

Parameters:

  • i1 int Index of first byte to remove, cf. MemoryBlob:GetBytes. (default 1)
  • i2 int Index of last byte to remove, ditto. (default #self)

Returns:

    uint Number of bytes actually removed.

See also:

MemoryBlob:Submit ()
(WIP) Submit this blob's contents to storage. This is a mechanism geared toward sharing memory among Lua processes, which can even be done without copying when certain conditions are met (cf. MemoryBlob:Sync for details, as well as the comments about resizable blobs that follow).

Essentially, an empty resizable blob with the same alignment is first created in storage.

If the original blob is resizable, its contents are swapped directly into the stored entry.

Fixed-size blobs cannot do this, so something like entry:Append(fixed_blob) is done instead.

To keep memory under control, any blobs in storage left unsynchronized will be evicted after a few frames have gone by (on average, 5). Listeners for "stale_entry" will be sent a message each time this happens (cf. GetBlobDispatcher), with an id field containing the former entry's ID.

This is a no-op for locked, resizable blobs.

Returns:

    string or nil On success, an ID for later use by MemoryBlob:Sync. Otherwise, nil.

See also:

MemoryBlob:Sync (id)
(WIP) Synchronize this blob with an entry in storage, which then ceases to exist.

For fixed-size blobs, this is essentially self:Write(1, bytes), with bytes being the entry's contents.

With resizable blobs, the same is true, but the blob's length is also trimmed to #bytes, if necessary. When the blob and stored entry have a common alignment (cf. MemoryBlob:Submit), the blob simply assumes ownership of the latter's contents, forgoing a potentially expensive copy.

This operation is designed for safe communication between Lua processes.

This is a no-op for locked blobs, or if id does not refer to a valid entry.

Parameters:

Returns:

    boolean Synchronization succeeded?

See also:

MemoryBlob:Write (pos, bytes)
Write bytes to the blob, starting at position pos, overwriting the blob's contents.

Resizable blobs will grow to accommodate these bytes, if necessary, whereas writes to fixed-size blobs will stop if they reach the end.

This is a no-op for locked blobs.

Parameters:

  • pos int Write position, between 1 and #self, inclusive. A negative index may also be provided, cf. MemoryBlob:GetBytes for details.

    Resizable blobs may also use position #self + 1, with behavior like self:Append(bytes).

    No bytes are written when the position lies outside the legal range.

  • bytes Bytes Bytes to write.

Returns:

    uint Number of bytes actually written.

See also:

generated by LDoc 1.4.0