Module `luaproc`

Corona binding for luaproc, a concurrent programming library for Lua.

Some additional features have been added, among them support for shared variables and events. The latter are built atop libcuckoo and pevents respectively.

To use the plugin, add the following in build.settings:

plugins = {
   ["plugin.luaproc"] = { publisherId = "com.xibalbastudios" }
 }

Sample code is available here and here.

Functions and sections that begin with (WIP) describe work in progress. These are features that have not made it into luaproc's public build, but give a reasonable idea of what to expect.

From the project page:

luaproc is a Lua extension library for concurrent programming. This text provides some background information and also serves as a reference manual for the library. The library is available under the same terms and conditions as the Lua language, the MIT license. The idea is that if you can use Lua in a project, you should also be able to use luaproc.

Lua natively supports cooperative multithreading by means of coroutines. However, coroutines in Lua cannot be executed in parallel. luaproc overcomes that restriction by building on the proposal and sample implementation presented in Programming in Lua (chapter 30). It uses coroutines and multiple independent states in Lua to implement Lua processes, which are user threads comprised of Lua code that have no shared data. Lua processes are executed by workers, which are system threads implemented with POSIX threads (pthreads), and thus can run in parallel.

Communication between Lua processes relies exclusively on message passing. Each message can carry a tuple of atomic Lua values (strings, numbers, booleans and nil). More complex types must be encoded somehow — for instance by using strings of Lua code that when executed return such a type. Message addressing is based on communication channels, which are decoupled from Lua processes and must be explicitly created.

Sending a message is always a synchronous operation, i.e., the send operation only returns after a message has been received by another Lua process or if an error occurs (such as trying to send a message to a non-existent channel). Receiving a message, on the other hand, can be a synchronous or asynchronous operation. In synchronous mode, a call to the receive operation only returns after a message has been received or if an error occurs. In asynchronous mode, a call to the receive operation returns immediately and indicates if a message was received or not.

If a Lua process tries to send a message to a channel where there are no Lua processes waiting to receive a message, its execution is suspended until a matching receive occurs or the channel is destroyed. The same happens if a Lua process tries to synchronously receive a message from a channel where there are no Lua processes waiting to send a message.

luaproc also offers an optional facility to recycle Lua processes. Recycling consists of reusing states from finished Lua processes, instead of destroying them. When recycling is enabled, a new Lua process can be created by loading its code in a previously used state from a finished Lua process, instead of creating a new state.

References

A paper about luaproc — Exploring Lua for Concurrent Programming — was published in the Journal of Universal Computer Science and is available here and here. Some information in the paper is already outdated, but it still provides a good overview of the library and some of its design choices.

A tech report about concurrency in Lua, which uses luaproc as part of a case study, is also available here.

Finally, a paper about an experiment to port luaproc to use Transactional Memory instead of the standard POSIX Threads synchronization constructs, published as a part of the 8th ACM SIGPLAN Workshop on Transactional Computing, can be found here.

From libcuckoo's license:

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Apache License 2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

From the license of CityHash (used by libcuckoo):

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

From pevents's license:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Functions

newproc (lua_code)	Create a new Lua process to run the specified string of Lua code.
newproc (f)	Variant of newproc that accepts a function.
alert (message, payload)	Dispatch an alert event (in the main state), e.g.
create_event ([manual_reset=false[, initial_value=false]])	(WIP) Create a new event for process scheduling.
create_integer (value)	(WIP) Create a shared integer variable.
create_number (value)	(WIP) Create a shared number variable.
delchannel (channel_name)	Destroy a channel identified by string name.
destroy_event (name)	(WIP) Destroy an event, if present.
destroy_integer (Name)	(WIP) Destroy a shared integer variable, if it exists.
destroy_number (Name)	(WIP) Destroy a shared number variable, if it exists.
estimate_concurrency ()	(WIP) Get an estimate of available hardware concurrency, e.g.
get_alert_dispatcher ()	Get the dispatcher used by alert, in order to add / remove listeners.
get_integer (name)	(WIP) Get a shared integer atomically, if present.
get_number (name)	(WIP) Get a shared number atomically, if present.
getnumworkers ()	Return the number of active workers (pthreads).
is_main_state ()	Check whether the running process is in the main state.
is_waiting ()	Check whether luaproc is waiting for processes to finish, either after a call to wait or when closing down.
newchannel (channel_name)	Creates a new channel identified by string name.
preload (name, loader)	(WIP) DOCME!
receive (channel_name[, asynchronous=false])	Receive a message (tuple of boolean, nil, number or string values) from a channel.
receive_allow_from_main ()	Variant of receive that allows unmatched receives from Corona's thread.
recycle (maxrecycle)	Set the maximum number of Lua processes to recycle.
reset_event (name)	(WIP) Reset an event.
send (channel_name, ...)	Send a message (tuple of boolean, nil, number or string values) to a channel.
send_allow_from_main ()	Variant of send that allows unmatched sends from Corona's thread.
set_event (name)	(WIP) Set an event.
setnumworkers (number_of_workers)	Set the number of active workers (pthreads) to n (default = 1, minimum = 1).
sleep (ms)	Put the current process to sleep.
update_integer (name, op, delta)	(WIP) Update a shared integer, if present.
update_integer_cas (name, op, expected, delta)	(WIP) DOCME!
update_number (name, op, delta)	(WIP) Update a shared number, if present.
wait ()	Wait until all Lua processes have finished, then continues program execution.
wants_to_close ()	Indicates whether Lua wants to close, either from trying to exit the app or Corona simulator, or on account of relaunching the Corona simulator.
wait_for_all_events (events, wait)	(WIP) Wait until all events in a group are set.
wait_for_any_events (events, wait)	(WIP) Wait until at least one event in a group is set.
wait_for_event (name, wait)	(WIP) Wait until an event is set.

Functions

newproc (lua_code)

Create a new Lua process to run the specified string of Lua code. The only libraries loaded initially are luaproc itself (available as the global luaproc) and the standard Lua base and package libraries.

The remaining standard Lua libraries (io, os, table, string, math, debug, and coroutine) are pre-registered and can be loaded with a call to the standard Lua function require.

In addition, system can be require()'d in a similar way. This contains approximations of the following Corona built-ins, under the same names: CachesDirectory, DocumentsDirectory, ResourceDirectory, TemporaryDirectory, getTimer(), and pathForFile().

A preload facility is available to load custom plugins in processes.

Finally, the following modules may also be required:

crypto, easing, json, lfs, lpeg, ltn12, mime, re, socket, sqlite3

Note that processes are unique Lua states, so Lua modules loaded in multiple processes, or in one process as well as Corona's own thread, will have different contents. Special care must be taken in the case of dynamically loaded native libraries, since these may keep some of their state outside Lua.

Thus, libraries meant to allow use in multiple processes should provide explicit guarantees and / or guidelines. For instance, a library that keeps no state at all is fine, as is one that only allocates memory via lua_newuserdata. Another approach would be to use thread-specific data.

(WIP, in probation)

If a process errors out, the manager will check whether it had a global named LUAPROC_ERROR. If one exists and is a table, its cleanup and report_method members are queried.

When cleanup contains a function, it is called as local result = pcall(cleanup, error), with error being the object—typically a string—left behind by the crash. This provides a means to restore consistency if the process was using non-trivial resources such as events. When result is non-nil, it replaces error.

The final error is handled according to report_method. By default, it gets printed to the console.

With the "alert" method, something like luaproc.alert("crash_report", tostring(error)) is performed, allowing the application to respond to the error. Example use cases include:

Sending a message such as "fatal", if it was a non-recoverable error.
Encoding the final process state into the error, then later attempt to resume or restart the operation.

When the method is "ignore", the error is silent.

Parameters:

lua_code string String of Lua code.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

Parameters:

f function Function.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

alert (message, payload)

Dispatch an alert event (in the main state), e.g. to react to something in a process.

Listeners for events named message will be alerted. The event contains payload in field payload.

In the main state, this will be performed immediately. Otherwise, alerts will fire during an enterFrame; if Lua is trying to close, this will do nothing.

Parameters:

message string User-defined message.
payload string, number, boolean or nil User-defined payload.

Parameters:

manual_reset bool Normally, waiting events are reset if and when the operation completes.

Events created with this flag are left as is, requiring explicit use of reset_event. (default false)
initial_value bool If true, the event begins in the set state, as if set_event had been called. Otherwise, the event begins as unset. (default false)

Returns:

string or nil

nil

Parameters:

value optional int Initial value. If absent, 0.

Returns:

string

Parameters:

value optional number Initial value. If absent, 0.

Returns:

string

Parameters:

channel_name string Name of communication channel.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

destroy_event (name)

(WIP) Destroy an event, if present.

This must not be called while waiting for the event.

Parameters:

name string Name of event, cf. create_event.

Returns:

boolean

Parameters:

Name string of number, cf. create_integer.

destroy_number (Name)

(WIP) Destroy a shared number variable, if it exists.

While not encouraged, this may be safely mixed with update_number operations on the variable.

Parameters:

Name string of number, cf. create_number.

estimate_concurrency ()

(WIP) Get an estimate of available hardware concurrency, e.g. for setnumworkers.

Returns:

uint

get_alert_dispatcher ()

Get the dispatcher used by alert, in order to add / remove listeners.

Calling this outside the main state results in an error.

Returns:

EventDispatcher

get_integer (name)

(WIP) Get a shared integer atomically, if present.

Parameters:

name string Name of integer.

Returns:

int or nil

nil

Parameters:

name string Name of number.

Returns:

number or nil

nil

Returns:

int

Returns:

boolean

is_waiting ()

Check whether luaproc is waiting for processes to finish, either after a call to wait or when closing down.

Returns:

boolean

luaproc

Parameters:

channel_name string Name of new communication channel.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

preload (name, loader)

(WIP) DOCME!

Parameters:

name string
loader optional function

receive (channel_name[, asynchronous=false])

Receive a message (tuple of boolean, nil, number or string values) from a channel. Suspends execution of the calling Lua process if there is no matching receive and the async (boolean) flag is not set. The async flag, by default, is not set.

Calling this synchronously from Corona's thread results in an error unless a corresponding send has already fired. As with send, this is usually desired, to avoid Corona blocking; receive_allow_from_main is provided to circumvent this protection.

Parameters:

channel_name string Name of communication channel.
asynchronous bool Receive asynchronously? (default false)

Returns:

send

Or

nil, indicating an error.
Error message, of type string.

receive_allow_from_main ()

Variant of receive that allows unmatched receives from Corona's thread.

recycle (maxrecycle)

Set the maximum number of Lua processes to recycle. The default number is zero, i.e. no Lua processes are recycled.

Parameters:

maxrecycle int Maximum number of processes.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

reset_event (name)

(WIP) Reset an event.

Parameters:

name string Name of event, cf. create_event.

Returns:

boolean

Parameters:

channel_name string Name of communication channel.
... Values to send, of the above-mentioned types.

Returns:

true

Or

nil, indicating an error.
Error message, of type string.

send_allow_from_main ()

Variant of send that allows unmatched sends from Corona's thread.

set_event (name)

(WIP) Set an event.

Parameters:

name string Name of event, cf. create_event.

Returns:

boolean

Parameters:

number_of_workers int Number of active workers.

Parameters:

ms optional uint Milliseconds to sleep (approximately), ≥ 0.

If absent or 0, the process yields, giving other processes a chance to use any spare time.

update_integer (name, op, delta)

(WIP) Update a shared integer, if present.

Concurrent updates are atomic (cf. the summary for op, below), but have no specific order.

Parameters:

name string Name of integer, cf. create_integer.
op string Binary update operation, which may be one of:
- "assign": assign delta directly to the variable.
- "add", "sub", i.e. arithmetic add, subtract.
- "and", "or", "xor", i.e. bitwise and, inclusive-or, exclusive-or.
The new value will be old op delta, e.g. new = old + 7 for op of "add" and delta of 7.

Atomicity means w.l.o.g. that when two processes are calling update_integer on the same integer at the same time, one of them will "win": effectively, one goes first and assigns its result to new, followed by the other using said result as its old.

If both processes saw the same old, there would be a "race" to set new, with unpredictable results.
delta int Value combined with the shared integer to update it, according to op.

Returns:

number or nil

nil

Parameters:

name string Name of integer, cf. update_integer.
op string As per update_integer.
expected int Expected value of integer.
delta int As per update_integer.

Returns:

true

expected

Or

false, meaning failure.
int or nil Actual value of integer when the operation was attempted (if absent, nil).

update_number (name, op, delta)

(WIP) Update a shared number, if present.

The additional details from update_integer apply here as well.

Parameters:

name string Name of number, cf. create_number.
op string Binary update operation, which may be "assign", "add" or "sub", cf. update_integer.
delta number Value combined with the shared number to update it, according to op.

Returns:

number or nil

nil

Returns:

boolean

wait_for_all_events (events, wait)

(WIP) Wait until all events in a group are set.

Parameters:

events {string,...} One or more names of events, cf. create_event.
wait optional uint Milliseconds to wait, ≥ 0. When absent, the wait will never time out.

N.B. At the moment, this does not respect Corona's suspend and resume logic.

Returns:

true, indicating success.
string First name in events.

Or

"timeout"

wait

Or

false

wait_for_any_events (events, wait)

(WIP) Wait until at least one event in a group is set.

Parameters:

events {string,...} One or more names of events, cf. create_event.
wait optional uint Milliseconds to wait, ≥ 0. When absent, the wait will never time out.

N.B. At the moment, this does not respect Corona's suspend and resume logic.

Returns:

true, indicating success.
string Name of set event from events with lowest index.

Or

"timeout"

wait

Or

false

wait_for_event (name, wait)

(WIP) Wait until an event is set.

Parameters:

name string Name of event, cf. create_event.
wait optional uint Milliseconds to wait, ≥ 0. When absent, the wait will never time out.

N.B. At the moment, this does not respect Corona's suspend and resume logic.

Returns:

boolean or string

wait

"timeout"

Otherwise, returns a boolean indicating whether the wait succeeded.

luaproc

Contents

Modules

Module luaproc

Functions

Functions

Parameters:

Returns:

Or

See also:

Parameters:

Returns:

Or

Parameters:

See also:

Parameters:

Returns:

See also:

Parameters:

Returns:

See also:

Parameters:

Returns:

See also:

Parameters:

Returns:

Or

Parameters:

Returns:

See also:

Parameters:

Parameters:

Returns:

Returns:

Parameters:

Returns:

See also:

Parameters:

Returns:

See also:

Returns:

See also:

Returns:

Returns:

See also:

Parameters:

Returns:

Or

Parameters:

Parameters:

Returns:

Or

Parameters:

Returns:

Or

Parameters:

Returns:

See also:

Parameters:

Returns:

Or

Parameters:

Returns:

See also:

Parameters:

See also:

Parameters:

Parameters:

Returns:

See also:

Parameters:

Returns:

Or

Parameters:

Returns:

See also:

See also:

Returns:

Parameters:

Returns:

Module `luaproc`