Capabilities & Limits

This describes all the capabilities and limits of PAC-1.

Maya uses the PAC-1 first-generation program synthesis engine to program itself from English instruction - planning, creating, replicating and modifying custom software tools to execute tasks across tools, applications, and APIs.

Here are some of its capabilities and limitations (as of March 2023):

Capabilities :

1. Learning from a few samples : The system can be taught how to do a new task by providing just 2 samples of the task as demonstration. It understands the similarities and differences between the samples, and uses that to predictably generalize to a similar instruction it hasn't seen. The more samples with variations provided, the more refined its understanding of that task.

2. Self-Referential : Maya can generate programs to call it's own API, making it possible to automate any task you can manually do using the Maya app like creating/calling new workers and generating/editing programs.

3. Long-term memory : Maya exposes an API to store, retrieve and answer questions from an internal vectorized document store. Programs have been taught to call this API, as a part of the Maya standard library, to do custom tasks like composing emails from a particular template, or storing documents from scraping the web to refer to later.

4. Turing-complete : The PAC-1 interpreter that converts natural language steps into executable programs is Turing-complete. Anything you can do with a scripting language like Python, can be done here as well.

5. Interpretable : For every natural language instruction, Maya generates a human-interpretable set of steps in plain English that it will execute. These steps can be modified or edited to change its intended behaviour. The PAC-1 interpreter converts these steps into executable graphs in our functional visual programming language, that are deployed to carry out the task.

6. Compositional : Each Maya step can be thought of as a standalone function that takes a message, processes it, and sends it over to the next step, like components along a factory line. Maya's functional behaviour can be changed by just rearranging these steps in different ways.

7. Deterministic : The PAC-1 interpreter is predictable and deterministic for unambiguous commands. Correct generations are globally cached for faster iteration times. For generations which need more subjective judgement, the cache can be over riden to regenerate the programs.

8. L1 Generalization : Today, Maya can robustly generalize only within the steps and recipes has been taught, or known variables within known domains.

9. Parallelization & Concurrency : Maya can launch new Worker runtimes to perform tasks in parallel (repeat the same task across N workers) or concurrently (break a task down into N steps and distribute it among N workers).

10. Edit by Instruction : Since the steps it generates are in natural language, they can be edited using just English instruction.

11. Continuous deployment : Once a program is generated, set up takes a few clicks (adding an API key, or authenticating via Oauth for third party tools), and then Maya deploys both the frontend and backend instantly. For subsequence edits, Maya detects and deploys only the changes, tightening the loop between instruction and final result.

Limitations :

1. Limited L2 Generalization : As of March 2023, PAC-1 cannot do robust L2 generalization, or generalize to unknown variables in known domains. This means it can automatically infer only short 3-4 step long combinations of steps it has been taught, if it hasn't been taught scripts to combine them before.

2. Search Collisions : Maya sometimes may mix up commands which look similar, like generate a research prompt vs generate an email prompt , or in the case duplicates have been taught. We're working on reducing the rate of such collisions + display the collision percentages while teaching.

3. Non-standard data interfaces : Most Maya steps are made to take in msg.payload process and output msg.payload. However some may have non-standard input and output types, for example might expect arrays instead of strings. In this case, refer to the documentation of the terminal nodes to get a better idea of how to use them.

4. Generation errors : Depending on the complexity of the step taught (specially for those that require subjective understanding), generated programs might be unsuited for use. You can either regenerate such a step to cycle through variations, or just manually change the generated program to match the expected behaviour.

5. Edit errors for longer, branched recipes : If your recipe has a lot of steps (20+) or lot of multiple indent levels, then edit instructions might fail to reflect.

6. Early research preview : PAC-1 is in early research preview, so components might break, or generations might be inaccurate. We hence recommend not using this for critical functions or data.