Many2One

20 September 2021 Link

On This Page



Description

Many2One is a utility to combine multiple lua source files into 1 file for easy usage and distribution. It consists of a lua file (many2one.lua) and an associated configuration file containing all the run information.
It will scan the given file and find all dependencies recursively using the the package.path and package.cpath settings in the Lua interpreter used to run many2one. So if you can run your program using the same Lua Interpreter on the same system, many2one will find all the dependencies and copy them over to make a self contained package. This saves a lot of time because earlier missing packages were found when the self contained package was executed.

NOTE: However the dependencies of the dynamic libraries are not followed or copied. That would be a feature for the future.

Example

Suppose you have an application with 2 files. main.lua which requires req.lua and their contents are:
main.lua:
  1. require("req")
  2.  
  3. a = 4
  4. print(a+b)
  5. print(c)

req.lua:
  1. local _G = G
  2.  
  3. local M = {}
  4. _ENV = M
  5.  
  6. _G.c=10


After running many2one with the config.lua of:
  1. -- Config.Lua file for many2one
  2. mainFile = "main.lua"
  3. outputFile = "MainApp.lua"
  4. deployDir = "deploy/"     -- directory should exist, it will empty the directory completely before creating the package

This will create the MainApp.lua file in the deploy directory.
The MainApp.lua file will look something like:
  1. do
  2.     local __MANY2ONEFILES={}
  3.     local reqCopy = require 
  4.     require = function(str)
  5.         if __MANY2ONEFILES[str] then
  6.             if not package.loaded[str] then 
  7.                 package.loaded[str] = true
  8.                 local res = load(__MANY2ONEFILES[str])
  9.                 res = res(str)
  10.                 if res ~= nil then
  11.                     package.loaded[str] = res
  12.                 end
  13.             end 
  14.             return package.loaded[str]         
  15.         else
  16.             return reqCopy(str)
  17.         end
  18.     end
  19.     __MANY2ONEFILES['req']="local _G = G\
  20. \
  21. local M = {}\
  22. _ENV = M\
  23. \
  24. _G.c=10"
  25. end
  26. require("req")
  27.  
  28. a = 4
  29. print(a+b)
  30. print(c)


Usage

Default configuration file config.lua

At the command prompt type:
> lua many2one.lua

This command opens up config.lua as the default configuration file that it expects in the same directory.

Specified configuration file

At the command prompt type:
> lua many2one.lua myconfig.lua

This command uses the configuration file myconfig.lua

The configuration file

A sample configuration file is shown below:
  1. -- Config.Lua file for many2one
  2. exclude = {
  3.     "llthreads",
  4.     "cURL"
  5. }
  6.  
  7. mainFile = "myscript.lua"
  8. outputFile = "myScriptApp.lua"
  9. deployDir = "../deploy/"


It's simple to understand. The main program file is added in the variable mainFile. The output file name is added in the variable outputFile. The exclude section is optional containing a list of modules that are found in the scripts that you don't want to search and include in the package. For example table module is packaged with lua but some programs sometimes do require("table"). So you can have "table" in the exclude list so you do not get the module not found error from the dependency searcher code.

And that is it.

Flags

mainFile

To specify the main program file, i.e. the application entry script file

outputFile

Name of the output file which will consolidate all the Lua scripts

deployDir

Path to the deply directory where the consolidated Lua Script and dependencies will be copied

txtExt

Table containing extensions to be used to consider as scripts if not using the internal default table containing "lua" as the extension.

moreExclude

List of libraries that you want to exclude included in the consolidation. Default ones already added are:
exclude = {		-- Any modules that need not be packaged
	"string",
	"table",
	"package",
	"os",
	"io",
	"math",
	"coroutine",
	"debug",
	"utf8"
}

include

List of libraries that you want to include in the consolidation even though they are not referenced anywhere in the code. For example if I want to add the penlight app and array2d libraries I would do:
include = {
     "pl.app",
     "pl.array2d"
}
So now in the code require("pl.app") will work if some external code uses it through your application.
Of course the included libraries should be available to many2one during consolidation.

clearDeployDir

Clears the deploy directory if set true othrwise does not.


How does it work

many2one takes the main program file and wraps the require function so that when require is called it first checked if the required file is present in the files packed in. If yes it returns the packed file otherwise it lets the original require function (stored in reqCopy) do the job.
  1. require = function(str)
  2.     if __MANY2ONEFILES[str] then
  3.         if not package.loaded[str] then 
  4.             package.loaded[str] = true
  5.             local res = load(__MANY2ONEFILES[str])
  6.             res = res(str)
  7.             if res ~= nil then
  8.                 package.loaded[str] = res
  9.             end
  10.         end 
  11.         return package.loaded[str]         
  12.     else
  13.         return reqCopy(str)
  14.     end
  15. end

It includes all the Lua files listed in the fileList of the config file as a string in the table __MANY2ONEFILES.

Dependencies

  1. submodsearcher - To modify the search paths to search properly for sub modules
  2. tableUtils - Useful table manipulation utilities
  3. LuaFileSystem - Lua file system module
  4. diskOP - Disk Operations library based on LuaFileSystem
  5. Logging - Logging library
  6. argparse - Command Line Argument Parsing


Download and Install

Download from the github repository. many2one is a single script file and if you have the above dependencies installed most of which (except luafilesystem) are simple Lua modules you can use it directly.
You can also install using LuaRocks:
luarocks install many2one
Once installed through LuaRocks you can directly run the script from the command line as:
>many2one.lua config.lua