This thesis focuses on the architecture of a data flow visual programming environment. An experimental system NeatTools was built to test and verify key concepts. Although the system and architecture considerations are general, software support for assisting interfaces is the main motivation for and test bed for our ideas.

The programming environment had to be implemented base on a programming language before the computer science experiments could proceed. In this thesis, I discuss the pros and cons between C/C++ and Java. Later on I discuss and compare NeatTools with Object Management Environments, Modeling Language, and some commercial visual programming tools.

Copyright 1999 Yuh-Jye Chang

All rights Reserved

1. Introduction *

2. Detailed Design Considerations of NeatTools *

3. Summary of NeatTools Implementation *

4. Experiments *

5. Conclusion *

6. Appendix *

7. Bibliography *

Figure 1: Top-level packages *

Table 1: Summary of layers *

1. Introduction

A workable human computer interface can be a vital asset for disabled users who may require computers for tasks such as data collection, gesture recognition, control of external devices, virtual world control, remote collaboration, and perceptual modulation. To accomplish this, a highly configurable, easy to use, extensible, and functional system is needed. In this thesis, the architecture of a fine-grained data flow visual-programming environment is proposed for experimentation and providing solutions to this challenge.

A lot of efforts have been emphasized on using visual programming to assist the traditional texture programming. For example, the commercial visual series products like Visual C++, J++, and Visual Basic from Microsoft Corp. [8] Others focus on creating the textural programming counterpart of visual programming such as G programming language that was used to develop the LabView [23] -- a graphical programming development environment. Or, focus on data visualization by using data-flow networks like AVS/Express [22]. The visual programming architecture in NeatTools is different from all that above. It is not intended to be a visual environment that helps generating the desired application template with textural code. It does not follow the traditional textural programming paradigm such as imitating the for-loop, while-loop, process structure, function call, etc. Nor does it emphasize on a particular field of application and try to integrate the coarse-grained modules through data-flow networks. NeatTools architecture is built strictly upon the module abstraction which has its roots in the formal Input / Output automation model of Lynch and Tuttle [1]. Module is the basic building block, which the modular communication is carried out through the logical connections. Base on NeatTools architecture, a generic fine-grained visual-programming environment is implemented from a pure visual programming and high performance perspective. All modules are logically equal and they all derive from the same module base class.

In the next section, I describe the detailed design considerations of NeatTools including project requirements, platform and language issues, architecture analysis, object management environments, modeling languages, and other visual programming tools. I discuss C++ vs. Java, C++ features that Java avoids, and features that Java adds. Also, I analyze NeatTools architecture and implementation concept through the module abstraction.

In section 3, I summarize the design goal of NeatTools to be simple, object-oriented, network-ready, robust, secure, architecture neutral, portable, high-performance, multi-threaded, and dynamic. I also introduce visual programming features such as multi-thread, keyboard/mouse event simulator/filter, networking, container nest structure, polymorph data type, multimedia, database, external module and state machine. Then, I discuss the NeatTools architecture including the three-layer architecture, package structure, OS and C++ runtime package, and Java-like API package, etc.

In section 4, the experiments were carried out through Eyal Sherman and Brooke Kendrick's case. I have described my approach to these cases, the limitations of early software, my ideas and improvements over the early software, and key issues in building better Neat software. Later, the experiments were presented and encouraging results were achieved.

2. Detailed Design Considerations of NeatTools
1. Project Requirements

1. General Human Interface Tool.
1. Can adapt to any external industry or in-house devices.
2. Able to handle complex, real-time, and calculate intensive tasks.
3. Support full multimedia functions. Enable users to control, manipulate and transform different multimedia data and information.
2. A visual programming environment.
1. Enable user to design and implement complete applications without the need to write any textual code.
2. Scalability and cross platform compatibility.

1. C++ vs. Java

NeatTools is an experimental computer science project. Before we can undertake any experiment, we need to have a tool or an environment. This makes it possible to build and test design concepts and proceed with the experiment itself. So, the first step was to implement such a visual programming tool. The question was which language to use to implement the tool so it would meet all the requirements and still have the potential for future development. C/C++ is the industry standard and Java is a rapidly developing and widely adopted new language. This section discusses the pros and cons of these two popular languages.

0. Java Background

Java is an object-oriented programming language developed by Sun Microsystems, a company best known for its high-end UNIX workstations. Modeled after C++, the Java language was designed to be small, simple, and portable across platforms and operating systems, both at the source and at the binary level.

Platform independence is one of the significant advantages that Java has over other programming languages. At the source level, Java's primitive data types have consistent sizes across all development platforms. Java's foundation class libraries make it easy to write code that can be moved from platform to platform without the need to rewrite it to work with another platform. Java binary files are also platform-independent and can run on multiple platforms without the need to recompile the source. How does this work? Java binary files are actually in a form called bytecodes. Unlike most other programming languages, The Java development environment has two parts: a Java compiler and a Java interpreter. The Java compiler takes the Java program and instead of generating machine code from source files, it generates bytecodes. To run a Java program, a user runs a program acting as a bytecode interpreter, which in turn executes a Java program. The user can either run the interpreter by itself, or for applets, there is a bytecode interpreter built into a Java-capable or enabled browser like Netscape [29] or Microsoft Internet Explorer [8] which runs the applet.

The disadvantage of using bytecodes is in execution speed. Because system-specific programs run directly on the hardware for which they are compiled, they run significantly faster than Java bytecodes which must be processed by the Java interpreter. In order to increase the performance of Java bytes, some companies are dedicated to accelerating the bytecode. JIT [25] (just in time) interpreter is one of the technologies that is widely adapted into commercial Web browsers. JIT interpreter usually loads the Java bytecodes, preprocesses it into system-specific code in a dynamical fashion, and then executes the system-specific code directly.

1. Benchmark

I wrote a simple Java benchmark program:

import java.applet.Applet;

 

public class benchmark extends Applet {

public int dummy(int i) { return i;}

public void start() {

int i, x;

x = 0;

System.out.println("Benchmark Start");

long b1 = System.currentTimeMillis(), b2;

for (i=0; i<100000000; i++) {

}

b2 = System.currentTimeMillis();

System.out.println("Empty For Loop "+(b2-b1));

b1 = b2;

for (i=0; i<100000000; i++) {

x += i;

}

b2 = System.currentTimeMillis();

System.out.println("For Loop with + and assign "+(b2-b1));

b1 = b2;

for (i=0; i<100000000; i++) {

x *= i;

}

b2 = System.currentTimeMillis();

System.out.println("For Loop with * and assign "+(b2-b1));

b1 = b2;

for (i=0; i<100000000; i++) {

x += dummy(i);

}

b2 = System.currentTimeMillis();

System.out.println("For Loop with function call and assign "+(b2-b1));

}

}

Translating the Java program into C++ would be:

#include <stdio.h>

#include <sys/timeb.h>

 

long currentTimeMillis() {

struct _timeb tstruct;

_ftime(&tstruct);

return tstruct.time*1000+tstruct.millitm;

}

 

class benchmark {

public:

int dummy(int i) { return i;}

void start() {

int i, x;

x = 0;

printf("Benchmark Start\n");

long b1 = currentTimeMillis(), b2;

for (i=0; i<100000000; i++) {

}

b2 = currentTimeMillis();

printf("Empty For Loop %d\n", b2-b1);

b1 = b2 = currentTimeMillis();

for (i=0; i<100000000; i++) {

x += i;

}

b2 = currentTimeMillis();

printf("For Loop with + and assign %d\n", b2-b1);

b1 = b2 = currentTimeMillis();

for (i=0; i<100000000; i++) {

x *= i;

}

b2 = currentTimeMillis();

printf("For Loop with * and assign %d\n", b2-b1);

b1 = b2 = currentTimeMillis();

for (i=0; i<100000000; i++) {

x += dummy(i);

}

b2 = currentTimeMillis();

printf("For Loop with function call and assign %d\n",

b2-b1);

}

};

 

void main() {

benchmark bm;

bm.start();

}

The test data are below (Test on PC with AMD-233MHZ CPU[30], Win95) :

Units (ms)	for loop 108	x += i	x *= i	x += dummy(i)
C++, VC++	0 *	880	1810	880
JDK 1.1 [25]	29000	50420	51300	110730
JDK 1.0	43830	82340	83210	170210
IE 4.0	42400	63330	63820	175550
IE 4.0 (JIT)	0*	930	2250	9010
Netscape 4.05	940	1810	2690	8510

* Optimized and eliminated by compiler or interpreter.

Units (us/loop)	for loop	x += i	x *= i	x += dummy(i)
C++, VC++	0 *	0.0088	0.0181	0.0088
JDK 1.1	0.29	0.5042	0.513	1.1073
JDK 1.0	0.4383	0.8234	0.8321	1.7021
IE 4.0	0.424	0.6333	0.6382	1.7555
IE 4.0 (JIT)	0*	0.0093	0.0225	0.0901
Netscape 4.05	0.0094	0.0181	0.0269	0.0851

* Optimized and eliminated by compiler or interpreter.

2. Benchmark Analysis

Based on the nature of object oriented programming, a function call is a must. Class interfaces are actually functions associated with objects. From the table above we learn that the most advanced commercial browser's Java interpreter is about 10 times slower than compiled C++ code when a function call is involved in the loop. And when the Java interpreter is not equipped with JIT, the ratio becomes unacceptably slower (around 200 times slower). And this is only a very simple benchmark program. When it comes to large applications, the JIT tends toward worse performance because it has a limited buffer for compiled code. If the whole application is too large to be compiled before it executes, the JIT interpreter will have to compile section by section on the fly and will thus slow down the execution speed. I estimate the JIT would run about 20 times slower than compiled C++ code when an application is relatively large. But, Java is changing and evolving rapidly compared to C/C++; JDK itself has gone through several changes and enhancements including JDK 1.1, JDK 1.2 [25], JDBC [25], Java 3D [25], JIT; and the long awaited Java compiler, HotSpot [25], will be available in 1999. A lot of experts in this field believe that it is possible that Java could increase in speed, performance, and eventually exceed what C/C++ does in the future. This means that the benchmark result today may not be similar in the future, but it could state the facts from the past until now.

3. Conclusion

When I started to design the next generation of the NeatTools project, I insisted on using C++ instead of Java. The main reason for that is speed and Java's uncertainty in the future. At that time, the JIT technology was not mature. Even today the JIT Java interpreter is still not fast enough for mission critical and calculation intensive real-time tasks like compress/decompress of voice and video data. However, like I discussed before, the benchmark result could only reflect Java as an immature technology at this time. It might not always be true in the future. Today, there are some other solutions such as linking native code into Java programs or using tools to convert Java bytecode into native code. But these alternatives are still not very efficient or stable. With the urgent need to help disabled people and solve other problems in the NeatTools project, my choice to use C++ was appropriate, because, I could then focus on the unique features and strategies for specific applications and experiments in NeatTools, not the language issue itself. In the future, when Java does become superior to C/C++, we may choose to rewrite NeatTools and adapt all the experience we have acquired towards implementing them in Java.

2. How to get JAVA without using Java

Here I would like to discuss some architecture issues. First, I will list the C++ features that Java avoids and investigate why they might be favorable. Actually, a programmer could benefit from those features when those features are well understood and used properly. Secondly, some features that Java adds could be useful. I will discuss how to implement the C++ counterpart and provide the desired functionality to the programmer.

0. C++ Features that Java avoids
0. Multiple Inheritance

Java provides single inheritance that is a subset of C++ multiple inheritance. Java class could "extends" from one super class and could "implements" multiple interfaces (abstract classes without any implementation and non-static data members). In C++, a class could inheritance from a set of classes including abstract classes (classes with one or more functions declared as abstract) or regular classes.

In theory, Java could provide the functionality of multiple inheritance by implements multiple interface classes. But in practice it could lead to code reusability problems. Lets look at the following example.

In Java 1.1, the new API provides a new mechanism that could make event broadcasting and listening more natural. This is a good use of object oriented programming. In the AWT class, the user usually extends from Canvas or Panel class to provide a windowing class with specialized functionality. In order to provide the basic action event broadcasting, the user usually has to implement the following methods like addActionListener(), removeActionListener(), and broadcastEvent(). The code is as follows:

ActionListener actionListener = null;

 

public void addActionListener(ActionListener l) {

actionListener = AWTEventMulticaster.add(actionListener, l);

}

public void removeActionListener(ActionListener l) {

actionListener = AWTEventMulticaster.remove(actionListener, l);

}

public void broadcastEvent(ActionEvent e) {

if (actionListener != null)

actionListener.actionPerformed(e);

}

 

The code above is simple and straightforward. Because not every windowing class needs to dispatch action events, these methods are not provided with the Component class (the super class for every windowing class). Now, there is no way for the Java programmer to reuse the code in different classes with different inheritance paths because of Java's single inheritance. For example, class A and class B are derived from Component and both classes need to broadcast action events. Then, both classes need to implement the exact same code segment and could not just inherit it from a separate interface (the interface does not carry implementation, only method declarations).

By using a different design pattern, one may design a separate class ActionBroadcast that can handle the event broadcast and include the class as a data member. That way we can wrap the methods, and the member class variable provides the actual implementation.

ActionBroadcast broadcast = new ActionBroadcast();

 

public void addActionListener(ActionListener l) {

broadcast.addActionListener(l);

}

public void removeActionListener(ActionListener l) {

broadcast.remvoeActionListener(l);

}

public void broadcastEvent(ActionEvent e) {

broadcast.broadcastEvent(e);

}

 

This solution solves the code reusable problem and, in the mean time, leads to an inefficient coding style. Because the method wrapper is always slower than a direct function call. When the underlying implementation is complex, the wrapper pattern will not account for too much performance loss. For the example above, this approach does not provide any benefit in terms of code reusability or efficiency.

1. Operator Overloading

In Java, operator overloading is eliminated completely. Though, the effects of operator overloading can be achieved by declaring a class, appropriate instance variables, and appropriate methods to manipulate those variables. I believe good use of operator overloading could actually increase the readability and sometimes the efficiency of the program.

For example, in some arithmetic classes like Vector, Matrix, and Coordinate, the use of operator overloading like addition, subtraction, multiplication, and division are very natural and could lead to a much more concise syntax. Without operator overloading, some complex calculations could become ugly and almost unreadable. Sometimes, one line of program expressed by variables and operators has to rewrite into several lines when expressed by variables and methods. This situation could cause the compiler to generate less optimized code and decrease efficiency.

Let's say we have Point3D as a 3D coordinate, and we want to calculate the point rotation with angle alpha. In Java, we will need to define the mul(double) add(Point3D) methods by the following statement:

Point3D p1 = new Point3D(1, 0, 0);

Point3D p2 = new Point3D(0, 1, 0);

p1.mul(Math.sin(alpha)).add(p2.mul(Math.cos(alpha)));

 

In C++, we define the operator*(double) and operator+(Point3D) methods and the statement becomes:

Point3D p1(1, 0, 0);

Point3D p2(0, 1, 0);

p1*sin(alpha)+p2*cos(alpha);

 

As you may notice, in C++, the statement is much shorter and easier to understand. Also, the notion of operators is natural to how we express mathematic equations. For complex Matrix or Vector operations, the statement in Java could become very long and unreadable. I believe, for a project group or for someone else to read the code, readability is a very important factor. In C++, we have the flexibility accomplish this by operator overloading or just by methods.

2. Pointer

The main reason Java eliminates pointer is that pointers are one of the primary features that enable programmers to inject bugs into their code. But, to my understanding, object oriented design could avoid class user's pointer usage completely. Also, good use of pointer could increase the efficiency and simplify the code generation.

In Java, the only way that a method could return primitive data type values in its parameters is through the use of Array.

int arg[] = new int[1];

foo(arg);

 

In C++, the same code could be expressed as:

int arg;

foo(&arg);

 

Obviously, the implementation in C++ is much better then Java. In Java, it requires more memory space (both reference for arg reference and int[] array class instance) and also it needs to implicitly initialize the internal Array class instance (which has unavoidable error checking code in Java API) for "arg" as an array of integers.

For those situations that pointer exposure is unwanted, we could use object class to hide the pointer and, in the mean time, provide the functionality needed. For example, to avoid user handle pointer directly, we could design IntArray class:

class IntArray {

int *data;

public:

IntArray(int size);

~IntArray();

int operator[](int index);

};

 

The IntArray class could encapsulate the integer pointer inside the class. It keeps the class user from using the pointer directly. In the mean time, it could allocate and de-allocate the memory properly in constructor and destructor. So, we don't have to worry about the pointer usage and memory allocation all together.

3. Memory Allocation

In the "Java white paper" [25], it states:

"Java completely removes the memory management load from the programmer. C-style pointers, pointer arithmetic, malloc, and free do not exist. Automatic garbage collection is an integral part of Java and its run-time system. While Java has a new operator to allocate memory for objects, there is no explicit free function. Once you have allocated an object, the run-time system keeps track of the object's status and automatically reclaims memory when objects are no longer in use, freeing memory for future use."

In theory, humans can always design better algorithms in compiler or interpreter to optimize the performance of high-level language. But Assembly language is still there to maximize the performance for some mission critical tasks in specialized computer industry. The most advanced-programming wizard could create quite a functional skeleton source code, but when it comes to specific requirements, an experienced programmer could still do a better job. Here, I try to express the idea that the human factor could never be eliminated in the decision making process. In the memory management case, according the different requirements and constraints, we can decide to implement a garbage collector in C++ or use object oriented programming to hide the memory management inside an object and distribute the memory management efforts into the object hierarchy. The programmer could use the class and never have to deal with the memory management problems directly. We can also decide to implement a reference count to eliminate unnecessary memory duplications, speed up the performance, and minimize the space requirement. In short, with proper set of classes support, C++ could become robust and free of memory leak problems. There are a great deal of different algorithms and implementation that we can choose in C++ or other object oriented programming languages that provide memory management primitives, but in Java we lose the right to choose. In C language's design spirits, it trusts programmers that they could do their job right. In Java, it only trusts the system designer. When a system designer fails to make good implementations, Java programmers have no choice but to use it anyway. For example, in Java3D, the API itself is still in beta test and has several bugs. One of the most complained about issues is that it consumes too much memory even when users only construct a fairly simply scene. I believe part of the reason lies in Java's memory management implementation. The garbage collector could automatically reclaim the unused memory. But the programmer's convenience comes with a price - we could nullify a class reference right away, but we don't know when it will be collected and reclaimed. For those big chunks of memory blocks (texture map, image buffer, or z buffer), they need to be claimed and reclaimed in the right order and in the right time. In this case, the garbage collector can not perform well and usually will cause memory usage problems.

4. Fragile Super Class

Fragile super class problems happen when a programmer modifies the header file of a base class; all the derived class which use this header file will need to be recompiled, or the executable linkage problem could occur. I believe this problem is inherited in the implementation of the current C/C++ compiler, not in the language itself. C/C++ compiler tends to parse the method name and data member identifier into an internal address. When the structure or the order of the identifier changes in the header file, the internal address and the lookup table change also. Without re-compiling the derived class's source code, the internal address of the methods or data members will not match. Thus, the fragile super class problem occurs - the programmer has to re-compile the super class before it could be linked to the derived classes. With a proper naming lookup protocol, this problem could be solved. On the other hand, a better way to solve the problem is to avoid changing the super class's interface. I believe, before the coding draft, a detailed class design and analysis will ensure a proper interface of a super class. Thus, we could avoid modifying the base class header file and prevent the fragile super class problem from happening.

1. Some Features that Java Adds
0. Build-in Thread and Synchronization Primitives

By using object oriented programming, multithreading capabilities could be added in C++ easily. The usage could be similar to what Java has. In the Thread class, we can have a class that uses Thread primitive (In Microsoft Windows, the Win32 API already has build-in thread support. In most UNIX systems, the pThread [31] is an add-on package to support the POSIX [31] thread) to create a new Thread and store its handle in the data member of Thread class. Later on when the thread is ready to start, we can redirect the thread process flow to the run() method. Thus, when user derives class from Thread class or Runnable class, it could then override the run() method and implement its thread loop. In "Java Like Cross Platform API," I follow the Java API standard and implement the JThread class in C++. It was proved that the API could hide the platform dependent implementation nicely, and programmers could have multithreading capabilities without noticing its underlying implementation differences.

To deal with concurrent problems, Java defines a new method modifier to integrate the thread synchronization into language. I believe this is a good move. But in C++, we can implement the mutex and critical section control into utility classes to help us handle the thread synchronization problems. In this case, we could have flexibility with how we want our process flow to be guarded by those synchronization primitives and make suitable adjustments to adapt to the needs of different situations.

1. Build-in Array

In Java, the Array class was created implicitly. The "new Data_Type[size]" syntax allows the user to create Array of any data type. It is very convenient, as there is no need to make type casting for the returned index elements. In the old-fashioned C++ design, the only way to make an Array is using the pointer of object pointer to implement the object array:

class ObjectArray {

int length;

Object **data;

public:

ObjectArray(int size);

~ObjectArray();

Object* operator[](int index);

};

 

Here, the polymorphism in C++ allows us to use ObjectArray to store any class that is derived from Object class. But the element returned by operator[] could only return a pointer of Object which needs to be cast before it could be used properly. Also, it could not store primitive data type and other classes which do not derive from Object class. So, in practice, it is not a good solution for a universal array.

Fortunately, C++ now includes a "template" which could embed the data type as a parameter into the class declaration and solve the problem for an easy usage universal array. The template could be declared as follows:

template <class DataType>

class ObjectArray {

int length;

DataType *data;

public:

ObjectArray(int size);

~ObjectArray();

DataType& operator[](int index);

};

 

Now, the same template could apply to any class and primitive data type as well. Also the elements return by operator[] will return the desired data type. So, the use of template could provide exactly the same functionality that Java's build-in array provides.

2. Cross platform and Windowing API

In C++, the windowing related task is never easy. I remember, in the early books that discussion of how to open a simple window and display "Hello World!" in any Window system was fairly complex. The program was usually more than 100 lines. Since C++/C was designed for environments with only text input/output, windowing was not natural and straightforward.

Later on, as the windowing environments became more and more popular, most of the C/C++ compilers began to provide the IDE (Integrated Development Environment) with graphical user interfaces running on a windowing environment. Also, compilers came with API (like OWL [32], MFC [8], and MOTIF [33]) that assist the programmer in creating their own graphical user interfaces. The problem is APIs are very different from one another. Also, for different windowing systems, there are all sorts of different APIs.

As the Internet becomes more prevalent, the interaction between different hosts running different operating systems and locations becomes more and more common and hence more and more important. Java was the first language, which supported cross platform development on the language level. Java's virtual machine and byte code system is the natural result of the language specification to be a cross platform language. I have discussed the cons and pros of this design. My solution to cross platform for C++ is an API that provides uniform support to basic utility and abstract windowing operations. It could encapsulate the underlying implementation, which is based on the windowing environment. So application dependency to windowing environment is eliminated, and the goal of cross platform in C++ could be easily achieved.

2. Conclusion

In this section, I examined the C++ features that Java avoids and discuss how to avoid the potential pitfalls. Instead of eliminating them, we could try to understand them, making good use of them and benefiting from those features. Overall, C++ is more flexible than Java. It provides multiple inheritance, operator overloading, and pointer. It could add code reusability, readability, and efficiency. For those userful features that Java adds, I propose a practical way to provide them in the C++. So C++ programmers could get Java's features without using Java. My conclusion is: object oriented programming in C++ is so powerful that it could be used to avoid commonly seen problems and also it could be extended with new kinds of functionality as well.

3. NeatTools Architecture Analysis

In NeatTools, the most important concept is module abstraction. The module abstraction simplifies and models all the interactions between modules into active actions and reactive actions. Later in this section, I will discuss the implementation concept and present some benchmark information and analyze it.

0. Module Abstraction

In NeatTools, module abstraction is offered as a set of class methods that provide inter-module communication functionality. Functional components (implemented as class objects) of a concurrent system are written as encapsulated modules that act upon local data structures, or objects inside object classes, some of which may be broadcast for external use. Relationships among modules are specified by logical connections among their broadcast data structures. Whenever a module updates data and wishes to broadcast the change, and make it visible to other connected modules, it should implicitly call an output service function which will broadcast the target data structure according to the configuration of logical connections. Upon receiving the message event, the connected modules execute an action engine according to the remote data structure. Thus, output is essentially a byproduct of computation, and input is handled passively, treated as an instigator of computation.

This approach simplifies module programming by cleanly separating computation from communication. Software modules written using module abstraction do not establish or effect communication, but instead are concerned only with the details of local computation. Communication is declared separately as logical relationships among the state components of different modules.

This programming model has its roots in the formal Input / Output automaton model of Lynch and Tuttle [1]. An I/O automaton is a state machine with a signature consisting of a set of input actions and a set of locally controlled actions. The locally controlled actions could divide into output actions and internal actions. Locally controlled actions are under the control of the automaton, while input actions may occur at any time. Automata may be composed such that when an output action of one automaton occurs, all automata having a same-named action as an input action may make a state transition simultaneously. A behavior of an I/O automaton is a sequence of input and output actions that may occur in an execution of that automaton. The module abstraction programming model is designed to benefit from the useful characteristics of the I/O automaton model that are helpful in reasoning formally about distributed systems.

Module abstraction is based on three fundamental concepts: data, actions, and connections. It is difficult to discuss these concepts in detail without reference to particular mechanisms for supporting them. Therefore, we present them in the context of NeatTools, a software run-time system and visual programming environment designed to support the development of data flow network applications using module abstraction.

0. Data

Data (the components of a module's state, could be data structures or objects) may be kept private, or they may be broadcast when needed so that other modules may access the data. NeatTools provides a base abstract class object that declares the basic data structure and service procedures (or methods). Every module object should inherit from this class object and override some predefined procedures to serve the different computational and presentational needs of each module. NeatTools provides a library of data types for declaring data that may be broadcast. These include but are not limited to integer, real, string, block, byte array, midi event, voice stream, and video stream. The module programmer may define others.

0. The Presentation

Each NeatTools module has a presentation that presents itself to the user as visual feedback. It could be graphical images, shapes, or text. The presentation may change dynamically according to the current state. Associated with each data item are a public name, property, access privileges, and data type. This information helps a user understand its presentation. The data type and access privileges also permit type check and privilege match of logical connections.

1. Access Privilege

Access privileges include input, output, insert, and connect. Output access allows connected modules to observe the value of the data when broadcast as an event message. Input access allows a module to change the state of the target module. Insert access allows a new connection to be inserted into an aggregate item of a module. Connect access allows a module to relate the data item to a data item of some other module. All those access privileges are controlled by a set of class methods (already defined in the base abstract class). By overriding those methods, modules could change the behavior or dynamically control access behavior according to the current state of the module.

2. The Desktop

A NeatTools module interacts with a desktop and a collection of other modules that may be unknown to this module but that read and modify the data item in its presentation when permitted by the access privileges. The desktop also works as a graphical user interface front end that provides a user with a set of layout service functions, including move, resize, copy, delete, group, ungroup, connection management, object persistency management and file input/output.

1. Actions

The action portion of a module defines how its state changes over time or responds to an environment. Insulated from the structure of its environment, a NeatTools module interacts entirely through the broadcast service and reactive execution engine procedures. A module may autonomously modify its local state, and also it may react to the incoming events and then change its local state. This suggests a natural division of the actions into two parts: active action and reactive action.

0. Active (Output) Action

The active action carries out the ongoing computation of the module. For example, in a discrete event simulation, the active action would be iterative computation that simulates each event. External updates of simulation parameters could affect the course of future iterations, but would not require any special activity at the time of each change. Modules with only active action can be quite elegant since input simply steers the active computation without requiring a direct response. Active action is analogous to the locally controlled actions of an I/O automaton.

1. Reactive (Input) Action

The reactive action carries out activities in response to input from the environment. A module with primarily reactive action simply reacts to input from the environment, updating its local state and presentation as dictated by the input change. For example, a data visualization module could be constructed so that each time some data element changes, the visualization is updated to reflect the change. In the above discrete event simulation, one might add reactive action to check the consistency of simulation parameters that are modified by the environment. Reactive action is analogous to the input actions of an I/O automaton.

2. Connections

Relationships between data items of different modules are declared with logical connections between those data items. These connections define the communication pattern of the system. Connections are established by a special NeatTools module, called a desktop, that enforces type compatibility across connections and guards against access protection violations by establishing only authorized connections.

Connections are declared separately from modules so that one can design each module with a local orientation and later connect them together in various ways. Connections are designed to accommodate all kinds of data types varying from integer and real number to audio and video. The run-time system could handle the communication requirements automatically according to the module abstraction of the module.

If we liken the data items of a NeatTools module to the actions in the signature of an I/O automaton, then just as like-named actions in automaton signatures define the sharing of actions, connections define the sharing of state change information. Currently, a simple synchronous data transmission algorithm is used by the broadcast service procedure. The reactive action engine of a connected module will be invoked and executed automatically. However, if asynchronous data transmission is needed, a user could construct a data queue inside the module and react to the data queue later. This way, we can keep the general communication structure simple and efficient.

1. Implementation Concept

The NeatTools' design concept is based on a special aspect of application design, which provides visual programming capacity. In traditional textual programming design, lines of code are the rough measure of program size. In a visual programming design like NeatTools' data flow networks, the number of connections and modules become the major indicators. Thus, how NeatTools provides efficient way to broadcast and process message events between modules becomes the most important issue. In NeatTools, all modules are already class objects. The only object interoperability is simplified into active and reactive actions, or connections. In order to increase the application throughput, the connections in NeatTools are just logical references. The message broadcasts are actually performed by direct function calls to ensure the performance of NeatTools' data flow network. Remote messages passing between NeatTools and a remote computer running NeatTools are performed by Socket and ServerSocket modules (just like regular modules, but they perform different tasks) to handle all the remote data communication tasks (usually on top of TCP/IP which is slow compared to a direct function call). Connections between Socket and other modules still use the same efficient function call. So, instead of a network connection oriented design (like CORBA [20]), NeatTools uses a direct function call oriented design that eliminates all possible overhead and layers.

2. Benchmark Information on NeatTools

I wrote a special benchmark module in NeatTools, which has two inputs and one output. The one on top can be enabled to start the benchmark. The one on the left can receive message events from other benchmark modules but do nothing. When the module is enabled, it will start the benchmark by broadcasting events through its output port in a 10⁶ loop and then display the time interval in the debug dialog box. The benchmark here is focused on the overhead of message broadcasting in NeatTools.

The engine method of benchmark module was implemented as follows:

void JBenchmarkObj::engine(int n, JLinkObj& link) {

if (!n) {

// link.access(JIntegerData(v[0]));

} else {

int oldv = v[1];

link.access(JIntegerData(v[1]));

v[1] = (v[1] != 0);

if ((v[1] != oldv) && !v[1]) {

int time = JSystem::currentTimeMillis();

for (int i=0; i<1000000; i++)

broadcast(0);

time = JSystem::currentTimeMillis()-time;

JComponent::debug(JInteger::toJString(time)+" ms");

}

}

}

The NeatTools benchmark data flow networks are arranged as follows:



The five test modules are instances of a benchmark module. One of them connects its enabled input from a button, and its output connects to the rest of the test modules.

3. Benchmark Analysis of NeatTools

The benchmark data for this particular data flow network running on my PC (AMD-233MHZ CPU, Win95) is 2200 ms, because the output has four connections and the broadcast processes are repeated for 10⁶ times. So 2200 ms / 4 / 10⁶ becomes 0.55 us per connection broadcast which is just 6.25 times slower compared to the direct function call with only one statement. In other words, NeatTools is capable of dispatching around 1,800,000 messages in one second. At this speed, I believe NeatTools could handle most real-time or computationally intensive tasks on the data flow network level.

4. Conclusion

The purpose of this section was to focus on how I proposed an abstract module model. Based on this model, I introduced a simplified and high performance way to implement the module connections, and how to map the connections into direct function call without much overhead or layers. This model proved successful as implemented in the NeatTools project. With the scalable design of the NeatTools project, I believe NeatTools could be widely adapted to most real world problems and provide complete and fast solutions.

4. NeatTools and Object Management Environments
0. CORBA

Object Managements Group [20] (OMG)'s Object Management Architecture (OMA) is the multi-vendor standard for object-oriented distributed computing. This includes CORBA -- the Common Object Request Broker Architecture -- which most people associate with OMG; CORBAservices and CORBAfacilities.

0. Object Definition

Objects are discrete software components -- they contain data, and can manipulate it. Usually, they model real-world objects, although sometimes it's useful to create objects specifically for items we want to compute. Other software components send messages to objects with requests; the objects send other messages back with their responses.

1. CORBA Provided Object Interoperability

In order for objects to plug and play together in a useful way, clients have to know exactly what they can expect from every object they might call upon for a service. In CORBA, the services that an object provides are expressed in a contract that serves as the interface between it and the rest of our system. The interfaces are expressed in OMG Interface Definition Language -- OMG IDL -- making them accessible to objects written in virtually any programming language, and the cross-platform communications architecture is the Common Object Request Broker Architecture -- CORBA.

2. OMA Provided Application-Level Integration

Based on CORBA architecture, the OMA specifies a set of standard interfaces and functions for each component. Different vendors' implementations of the interfaces and their functionality then plug-and-play on customers' networks, allowing integration of additional functionality from purchased modules or in-house development.

The OMA is divided into two major components: lower-level CORBAservices and intermediate-level CORBAfacilities. The CORBAservices provide basic functionality that almost any object might need: object lifecycle services such as move and copy, naming and directory services, and other basics. The CORBAfacilities architecture has two major components: one, horizontal, including facilities such as compound document services which can be used by virtually every business; and the other, vertical, standardizing management of information specialized to particular industry groups.

1. COM/DCOM

COM [8] (Component Object Model) is a software architecture that enables a program to be built from smaller binary components. It is a binary standard for component interoperability and is independent of any programming language.

COM supports a client/server model between the user of some object's services and the implementers of that object and its services. COM's role is to establish the connection between the client and the server, which offers the desired object. Once the connection has been made, COM is out of the picture and all communication goes directly from server to client and vice versa.

0. Objects and Interfaces

In COM an object is an instance of a class which as in standard OO terminology is a set of data and related functionality. Unlike most other OO models, COM provides no direct access to object data. Instead, users have to access member functions of an associated interface.

An interface is a set of functions that can be invoked on a given object. Interfaces do not contain any implementation what so ever, but merely define the expected behavior of an object. When an object implements an interface, it provides implementations for every function in that interface, and provides pointers of those functions to COM.

1. Interface Description Language (IDL)

COM's Interface Description Language (IDL) is based on the Open Software Foundation (OSF) [33] Distributed Computing Environment (DCE) [33] specification for describing interfaces, operations, and attributes to define remote procedure calls.

A designer can define a new custom interface by writing an interface definition file. The interface definition file uses the IDL to describe data types and member functions of an interface. The interface definition file contains the information that defines the actual contract between the client application and the server object.

2. Service Control Manager (SCM)

The SCM is a COM component that is able to locate a given server and launch it. The SCM contains a database of class information. When a client requests the COM library to create an object, the SCM is launched, the server located and run. Here SCM provides the object interoperability.

2. A Comparison of CORBA and COM/DCOM

From the communication point of view, COM is a little bit different from CORBA. In CORBA, ORB is always the gateway between its object client and server. The client and server in CORBA never communicate with each other directly. In COM, SCM locates and launches the object server for the application client, and then the client communicates directly with the server. Also COM has different mechanisms for in-process, cross-process, and remote object servers. In in-process cases, the SCM will locate and load the object server as a DLL. So in this situation, the object server will be executed in the client's process space, which is much faster than CORBA's mechanism. In cross-process cases, the SCM will locate and load the object server as an executable. The object server will be executed in a separate process space. So it is slower than the DLL but still faster then the CORBA's mechanism. Even in remote object server cases, the SCM will contact the remote SCM and later on build a remote proxy server which can forward requests directly to the remote SCM via the RPC connection. So, in general, COM/DCOM has better performance than CORBA. But CORBA has better design based on system and structured concerns. For example, a CORBA's object server will only deal with its local ORB. But a DCOM's object will have to deal with local SCM directly, or in-process client, cross-process client, and remote client through an interface wrapper.

From the application point of view, COM is lacking in application integration packages. CORBA's CORBAservices and CORBAfacilities components provide application-level integration which in turn provides lower-level and intermediate-level services for industry and business applications.

3. Benchmark Information on CORBA

The most important advantage that Object Management Environments like CORBA, and COM/DCOM is object interoperability. Object interoperability allows objects, which plug and play together in a useful way to become possible. But the advantage comes with a price -- speed. For example, in CORBA, when an object client issues a function call to a remote server object, it has to go through the IDL Stub (created by IDL), Object Request Broker (ORB), remote ORB, IDL Skeleton, and finally reaches the remote server object implementation. And in most cases it goes through a network communication layer like TCP/IP at least twice (forward the request and send back the result message) to complete a function call.

I visited one of the ORB vendor sites which has a web page dedicated to performance information. The URL for this page is http://www.orl.co.uk/omniORB/omniORBPerformance.html. From this page, came one of the tables below:

 

Performance of omniORB2 on various platforms.

Platform	Transport	Protocol	us/call
LINUX Pentium Pro 200 MHz	IP/intra-machine	IIOP	340
	IP/ethernet (ISA)	IIOP	1000
	IP/ATM	IIOP	440
	AAL5/ATM	IIOP	350
Windows NT Pentium Pro 200 MHz	IP/intra-machine	IIOP	360
	IP/ethernet (ISA)	IIOP	1000
Digital UNIX 3.2 Alpha DEC 3000	IP/intra-machine	IIOP	750
	IP/ethernet	IIOP	1050
Windows '95 Pentium 166 MHz	IP/intr-machine	IIOP	1000
	IP/ethernet (PCI)	IIOP	1250
SOLARIS 2.51 Ultra 1 167 MHz	IP/intra-machine	IIOP	540
	IP/ethernet	IIOP	710

4. Benchmark Analysis of CORBA

From the table above, to make a function call, even if the function call is made within the machine (the IP/intr-machine), it takes about 360 us on a Pentium Pro 200MHz machine. If we compare the benchmark information with the previous section, making a direct function call in compiled C++ code only takes 0.0088 us which is about 41,000 times faster than making function calls on this particular ORB. For those calls that are made remotely by a direct network card connection (the IP/ethernet), it takes 1,000 us to make a function call. In that case it is about 114,000 times slower than the compiled C++ code.

5. Conclusion

CORBA and COM/DCOM provide object interoperability, which is a great feature for object plug and play and object reusability. In CORBA, it especially tries very hard to separate the object server from object client to ensure encapsulation and increases the object interoperability by using ORBs. But this approach comes with a big draw back -- decreased performance to a great degree. In COM/DCOM, the performance problem is less severe. Because COM/DCOM provides direct access to the object server when an object server is loaded as an in-process DLL. But, even though COM/DCOM is independent of the programming language, some of the higher level control mechanisms involve registry operations in Microsoft Windows like Win95 or NT which we try to avoid in NeatTools due to cross platform concerns.

It is possible that for the NeatTools project to be built upon CORBA architecture. A NeatTools module designer will be able to put the modules on line and allow user remote access. But, NeatTools will have to limit data flow design with modules that do not require a lot of message event passing. For those fine-grained modules such as digital logic, number, and real number operation modules, decreasing the message passing speed will greatly hurt the performance. In this case, we won't be able to build data flow networks in NeatTools complex enough to meet the requirements of a general human interface tool. Other than the performance concern, every component in CORBA or COM/DCOM needs to dedicate some efforts and space on the interfacing, because the interfacing is where the object interoperability comes from. NeatTools has simplified broadcast models, as the interface is actually fixed and derived from the JModuleObj class. That means we have to dedicate identical interfacing by using IDL for each module, which is actually a redundant layer and could be eliminated. Thus, based on the space and speed constraints, I would suggest not using CORBA or COM/DCOM to develop NeatTools.

5. NeatTools and Modeling Languages
0. Colored Petri-Net

The following material regarding Petri-Net is extracted from the web page at the URL http://www.daimi.aau.dk/PetriNets/.

0. Introduction

Colored Petri-Net (CP-nets or CPN) is a graphical oriented language for design, specification, simulation and verification of systems. It is in particular well-suited for systems in which communication, synchronization and resource sharing are important. Typical examples of application areas are communication protocols, distributed systems, imbedded systems, automated production systems, work flow analysis and VLSI chips.

1. Why use CP-nets?

CP-nets are used for three different - but closely related - purposes. First of all, a CP-net model is a description of the modeled system, and it can be used as a specification (of a system to be built) or as a presentation (of a system to be explained to other people, or ourselves). By creating a model, we can investigate a new system before we construct it. This is an obvious advantage, in particular for systems where design errors may jeopardize security or would be very expensive to correct. Secondly, the behavior of a CPN model can be analyzed, either by means of simulation (which is equivalent to program execution and program debugging) or by means of more formal analysis methods (which are equivalent to program verification). Finally, it should be understood that the process of creating the description and performing the analysis usually gives the model designer a dramatically improved understanding of the modeled system -- and it is often the case that this is more valid than the description and the analysis results themselves.

2. Analysis of CP-nets

CP-nets can be analyzed in three different ways:

The first analysis method is simulation. It is very similar to debugging and program execution. This means that we can execute a CP-net model (e.g., to get statistics about the behavior of the modeled system). It is possible to set breakpoints and to display the simulation results by means of different kinds of business graphics.

The second analysis method is occurrence graphs (also called state spaces or reachability graphs). The basic idea behind occurrence graphs is to construct a directed graph, which has a node for each reachable system state and an arc for each possible state change. Obviously, such a graph may become very large, even for small CP-nets. However, it can be constructed and analyzed totally automatically, and there exist techniques which makes it possible to work with condensed occurrence graphs without losing analytic power. These techniques build upon equivalence classes.

The third analysis method is place invariants. This method is very similar to the use of invariants in ordinary program verification. The user constructs a set of equations, which provides the satisfaction of all reachable system states. The equation is used to prove properties of the modeled system (e.g., absence of deadlock).

3. Design/CPN

Design/CPN is a tool package supporting the use of CP-nets. The Design/CPN tool is now distributed free of charge to all kinds of users (including commercial companies). Version 3.0 was released in May 1996. It has three integrated parts:

1. The CPN Editor supports construction, modification and syntax check of CPN models.
2. The CPN Simulator supports interactive and automatic simulation of CPN models.
3. The Occurrence Graph Tool supports construction and analysis of occurrence graphs for CPN models (also known as state spaces or reachability graphs/trees).

Design/CPN supports CPN models with complex data types (color sets) and complex data manipulations (arc expressions and guards) - both specified in the functional programming language Standard ML. The package also supports hierarchical CP-nets (i.e., net models that consist of a set of separate modules with well-defined interfaces).

1. Conclusion

From what I have observed, CP-nets are a good tool to simulate and verify software or hardware projects. But according to one of the Design/CPN introduction articles on the web, Design/CPN could do around 1,000 message broadcasts per second, which compared to NeatTools' 1,800,000 messages per second is around 1,800 times slower. As a simulation tool, it is OK to have a speed at this level. For real-time applications, Design/CPN may not be the proper choice. The key difference is in the broadcast model itself not implementation. CP-nets use asynchronous message broadcast between modules as the default. It could avoid the cyclic deadlock problem and be much like real world conditions. But in the mean time, it decreases the performance a lot. NeatTools uses synchronous message broadcasting and simplifies the broadcast into a direct function call. Also, it uses a simple mechanism to avoid the cyclic deadlock problem without a great deal of overhead.

In "An Introduction to the Practical Use of Colored Petri Nets," by Kurt Jensen, [19], he gives the following description:

In my opinion, actions and states could all be expressed and manipulated by message events, because message events could stand for expressions of current states or actions. Actually, in NeatTools, I did implement the state as a module into NeatTools. The user could layout state machine visually and use it for complex state analysis. CP-net's decision to separate the modules into states and actions may have some advantage for theory analysis. But it could make the model itself become more complex and hard to understand for those users who have no formal theory analysis training.

1. UML (Unified Modeling Language)

The following material regarding UML is extracted from Rational's [21] web page at http://www.rational.com/ and OMG's [20] web page at http://www.omg.org/.

0. Introduction

The Unified Modeling Language (UML) is a language for specifying, visualizing, constructing, and documenting the artifacts of software systems, as well as for business modeling and other non-software systems. UML represents a collection of the best engineering practices that have proven successful in the modeling of large and complex systems.

1. UML definition

UML definition consists of the following documents:

1. UML Semantics: defines the rich semantics and expressive syntax of the Unified Modeling Language. UML is layered architecturally and organized by package. Within each package, the model elements are defined in terms of their abstract syntax (using UML class diagram notation), well-formedness rules (using text and Object Constraint Language expressions), and semantics (using precise text). Two appendices are included: UML Glossary and Standard Elements.
2. UML Notation Guide: defines notion and provides supporting examples. UML notation represents the graphical syntax for expressing the semantics described by the UML metamodel.
3. UML Extension for the Objectory Process for Software Engineering and UML Extension for Business Modeling: These UML extensions include process-specific and domain-specific extensions to UML, in terms of its extension mechanisms and process-specific diagram icons.

UML uses OCL, defined separately in the Object Constraint Language Specification document.

1. Development Project Artifacts

The choice of what models and diagrams one creates has a profound influence on how a problem is attacked and how a corresponding solution is shaped. Abstraction, the focus on relevant details while ignoring others, is a key to learning and communicating. Because of this :

1. Every complex system is best approached through a small set of nearly independent views of a model; No single view is sufficient.
2. Every model may be expressed at different levels of fidelity.
3. The best models are connected to reality.

In terms of the views of a model, UML defines the following graphical diagrams: use case diagram, class diagram, behavior diagrams, and implementation diagrams. These diagrams provide multiple perspectives of the system under analysis or development. The underlying model integrates these perspectives so that a self-consistent system can be analyzed and built.

1. Programming Languages

UML, a visual modeling language, is not intended to be a visual programming language, in the sense of having all the necessary visual and semantic support to replace programming languages. UML is, however, a language for visualizing, specifying, constructing, and documenting the artifacts of a software-intensive system, but it draws the line as you move toward code. Some things, like complex branches and joins, are better expressed in a textual programming language. UML does have a tight mapping to a family of OO languages, so that the user can get the best of both worlds.

2. Conclusion

Since UML is not intended to be a visual programming language, it is very clear that it has no conflict with the NeatTools application and its design concepts as a visual programming environment. On the contrary, we can use UML to help in specifying, constructing, and documenting the artifacts of the development of the NeatTools system or module design. The Rational Rose, a visual modeling tool that allows developers to define and communicate a software architecture, could be used to speed up the development of future module design for NeatTools. In this sense, UML could indeed be very useful to NeatTools development.

1. NeatTools and Other Visual Programming Tools
0. Microsoft Visual Series on C++, Basic, J++

The following material regarding Microsoft [8] Visual Series is extracted from Microsoft's web page at http://msdn.microsoft.com/visualj/, and http://msdn.microsoft.com/visualc/, and http://msdn.microsoft.com/vbasic/.

0. Introduction

Microsoft Visual Series uses the same Developer Studio environment that comes with Microsoft's Visual C++. Microsoft has done a good job incorporating its existing tools into VJ++ and Visual Basic. For example, to create dialog boxes, developers use the same dialog box editor found in VC++'s Developer Studio.

Windows developers will like the COM integration in VC++, VB, and VJ++. COM, Microsoft's component object model, is the core of ActiveX, and developers can incorporate existing ActiveX controls and COM-based technologies into their VC++, VB and VJ++ applications.

Developers can also build their ActiveX controls for use with Visual Basic, VC++, and Delphi. This is a fairly complex process, but the final release of VJ++ is expected to include an ActiveX Wizard that will make it easier to convert applets into ActiveX controls.

Developers could use application (or applet in VJ++) Wizard to create a small application. After they respond to the Wizard's few simple questions, it creates a commented source-code skeleton (and an HTML file in VJ++) to host the application (or applet). Thus, a user could start from there without the need to start from scratch.

1. Conclusion

Basically, I would consider Microsoft's Visual Series as a textual programming environment with some visual tools (like dialog box editor), ActiveX Wizard, and application Wizard to help a programmer integrate their applications with resources, ActiveX components, or to create skeleton source-code, etc. Conceptually, this is not relevant to the visual programming environment that NeatTools provides.

1. JavaBeans

The following material regarding JavaBeans is extracted from JavaSoft's [24] web page at http://www.javasoft.com/beans/index.html.

0. Introduction

The goal of JavaBeans is to define a software component model for Java, so that third party ISVs can create and ship Java components that can be composed together into applications by end users.

What is a Java Bean? A Java Bean is a reusable software component that can be manipulated visually in a builder tool. The builder tools may include a web page builder, visual application builders, GUI layout builders, or event server application builders. Or it may simply be a document editor that includes some beans as part of a compound document.

Some JavaBeans components may be simple GUI elements such as buttons and sliders. Other JavaBeans components may be sophisticated visual software components such as database viewers, or data feeds. Some JavaBeans components may have no GUI appearance of their own, but may still be composed together visually using an application builder.

Individual JavaBeans components will vary in the functionality they support, but the typical unifying features that distinguish a JavaBeans component are:

1. Support for "introspection" so that a builder tool can analyze how a bean works.
2. Support for "customization" so that when using an application builder a user can customize the appearance and behavior of a bean.
3. Support for "events" as a simple communication metaphor that can be used to connect up beans.
4. Support for "properties", both for customization and for programmatic use.
5. Support for persistence, so that a bean can be customized in an application builder and then have its customized state saved away and reloaded later.

While beans are primarily targeted at builder tools, they are also entirely usable by human programmers. All the key APIs such as events, properties, and persistence, have been designed to work well both for human programmers and for builder tools.

1. Linkage Model in JavaBeans

The linkage model in JavaBeans is naturally built on top of new Java 1.1 event broadcast mechanism and object class runtime information API. A bean programmer should follow proper design patterns to design interface method and property signature. So, the bean builder could study a bean's supported properties, events, and public methods through a low-level reflection mechanism or via BeanInfo class. In order to broadcast events between beans, bean builder first obtains the source bean, destination method, and event type information through its user interface. Later, it has to create a linkage class dynamically. The linkage class will register itself as an event listener to the source bean and also acts as a bridge between the event source and destination method. So, when the source bean broadcast an event, the destination method will be invoked through the linkage class. The linkage building process does not happen completely in run-time, because the dynamically created linkage class and other generated source files needs to be compiled before the application could be executed.

2. Conclusion

The most noticeable differences between NeatTools' module model and JavaBeans' component model are introspection process and linkage creation.

JavaBeans use a low-level reflection mechanism or BeanInfo class to study the methods supported by a target bean. In Java, the low-level reflection mechanism is already built in the system API. For example, the object.getClass method returns a class object, which describes the current object instance. Through the class object, we could walk the class hierarchy or determine the signature of a particular method. In C++, this kind of mechanism is almost impossible without an extensive implementation to embed the detailed run time class object information automatically. In NeatTools, the introspection process is not needed, because methods signature are fixed and inherited directly or indirectly from the abstract module base class. By deriving class from a generic data type class, methods with fixed signature could be used to broadcast events with different data types. Hence, the same functionality could be achieved.

In NeatTools, the linkage creation process is very simple and efficient compared to JavaBeans. The linkage data structure is fixed and it records the source module, source port number, destination module, and destination port number. No linkage class creation and compilation is involved during the linkage creation process. When a user requests to create a connection by mouse drag and drop, the NeatTools desktop just initializes a linkage data, fills in the information, and adds it into the linkage set. Unlike NeatTools, bean builder usually does not have a visual linkage layout representation. Instead, the linkage information between beans is accessed through the pop up menu or separate data entry form.

In general, JavaBeans' linkage and event broadcast model is very flexible and scalable. It takes advantage of some special features inside Java API, which is missing in C++, to assist the introspection process. In stead of using run time object class information, Object oriented programming technique was used to implement the module abstraction and event broadcast, which is simpler and more efficient. Since we decide to implement NeatTools in C++ and JavaBeans' linkage model is not suitable for language other than Java, NeatTools' module abstraction and event broadcast architecture is the reasonable implementation choice.

1. LabView

The following material regarding the LabView software package is extracted from National Instruments' [23] home page at http://www.natinst.com/.

0. Introduction

LabView is a graphical programming development environment based on the G programming language for data acquisition and control, data analysis, and data presentation. LabView gives a user the flexibility of a powerful programming language without the associated difficulty and complexity because its graphical programming methodology is inherently intuitive to scientists and engineers.

With LabView, a user builds VIs (virtual instruments) instead of writing programs. Users can create front panel user interfaces, giving them interactive control of the software system. To specify the functionality, a user assembles block diagrams - a design notation for engineers and scientists.

1. Data flow Programming

LabView implements a patented data flow programming model, called G, that frees the user from the linear architecture of text-based languages. Because the execution order in LabView is determined by the flow of data between blocks, and not by sequential lines of text, users can create diagrams that have simultaneous operations. LabView is a multitasking and multithreaded system, running multiple execution threads and multiple VIs.

2. Graphical Compiler

In many applications, execution speed is critical. LabView provides the compiler that generates optimized code with execution speeds comparable to compiled C programs. With the build-in Profiler, a user can analyze and optimize time-critical sections of code.

3. Multithreading

In the new version of LabView, the multithreading is built into VI, or LabView programs, thus it is not necessary for users to learn new programming techniques. In fact, the user can benefit from multithreading without even knowing what it is. However, for expert users who want to have specific control over threads, such as changing thread priorities, that flexibility is available in a straightforward dialog box option.

4. Conclusion

NeatTools and LabView are both visual programming environments. NeatTools is based on the abstract module model and LabView is based on the G programming language. The abstract module model tries to generalize and simplify the module and event broadcast model from a pure visual programming perspective, while the G programming language tries to have the graphical representation of traditional textual programming. For example, you can see the "while loop" and "for loop" structures in G programming language. In NeatTools, we do not define "while loop" or "for loop" structure in module specification. All modules are logically equal and they all derive from the same module base class. In one word, NeatTools users could use the Timer or ClockDivider modules to provide the same functionality. Timer and ClockDivider are just regular modules. Basically, a NeatTools user could design and use anything he likes to accomplish the desired tasks.

The graphical compiler in LabView provides optimized code with execution speeds comparable to compiled C programs. This indicates that the simulated version in LabView is slower and much bigger than the compiled version. In NeatTools, as I discussed in previous sections regarding NeatTools' performance, its data flow network is already comparable to compiled C/C++ programs. Also, NeatTools executables only include the core modules, thus the user could just add functionality by including the external module DLLs needed to meet the space constraints. Hence, the compiler is actually not needed in NeatTools.

Recently, the new LabView version started to support multithreading. The thread itself is associated directly in the VI. A user can change the thread's priority through a dialog box. I do not think this kind of add-on threading could provide a user all the power and flexibility of threading. In NeatTools, every module could start their own thread (or multi-threads) depending on the tasks it needs to perform. By using connections, several modules could share threads together, or modules could use internal threads for calculation intensive tasks, etc.

LabView has a very strong user group and history going back over ten years. NeatTools originated about 2 years ago. NeatTools has some technology advantages over LabView owing to its fairly new structure and design. I believe, with proper promotion, NeatTools could become popular in the educational market as well.

2. AVS/Express

The following information regarding AVS/Express is extracted from the home page of Advanced Visual System Inc. [22] at http://www.avs.com/.

0. Introduction

AVS/Express is a tool for developing data visualization applications. It enables users to explore data sets of any dimension or complexity using visualization capabilities.

AVS/Express offers the same benefits as other advanced client/server development tools. It provides a multi-platform, multi-OS development environment that allows broad-based application building. It enables developers to spend their time adding value to applications instead of trying to solve non-productive, costly system-level issues.

1. The Graphics Display Kit

The AVS/Express Graphics Display Kit is a set of objects that contain the data structure and function to develop data-intensive, interactive graphics applications. It provides the following graphics-component technology essential for rendering and manipulating text, 2D images, and 2D and 3D geometric objects. These components can be reconfigured, customized and easily replaced to provide custom views for end-user applications.

2. The Data Visualization Kit

The Data Visualization Kit contains objects, data structures and libraries needed to visualize and analyze data sets. The data model within this kit defines how data is represented, how functions access it, and how it is communicated between functions and processes. The AVS/Express data model could handle the data frequently found in data and graphics intensive applications, including image, volume, finite element, scattered and geometric data. The visualization components in this kit contain computation methods that are key to turning abstract data sets into graphics.

3. The Database Kit

The Database Kit provides interfaces to the most popular SQL-based relational database management systems (RDBMS), including Oracle [15], Sybase [35] and Informix. In addition, the data kit contains an Open Database Connectivity (ODBC) interface. A library of configurable visual objects provides connections to one or more relational databases, and enables display of database tables and assembly of SQL expressions.

4. Visual Programming

In AVS/Express, users construct applications as a connected, hierarchical network of objects, create and modify data structures with "drag-and-drop" ease, manipulate widgets directly to create GUI layouts, set up GUI callbacks through visual connections between widgets and application methods, examine application state and data with integrated debugging tools, and integrate external functions with encapsulated C, C++ and FORTRAN routines.

5. Conclusion

NeatTools is meant to be a generic fine-grained visual programming tool for developing human computer interfaces. NeatTools' focus is on how we could increase the information flow throughput between humans and computers. Thus, the key issues are on how we provide more information paths from humans to computers through sensors, devices, and broaden all kinds of feedback from computers to humans through graphics, audio, video, touchware, etc. It is possible that we could write all kinds of AVS/Express modules to provide the functionality specific to the NeatTools project. But, we never could change AVS/Express system itself to meet some special constraints and solve the problems specific to the NeatTools project. Due to different design concepts and implementation architectures, AVS kernel provided the centralized control over the inter-modular communication, and hence serialized parallel channels. Also, AVS uses IPC data-flow protocol to implement the channel event broadcast and data exchange, which limits AVS as a coarse grain visual programming tool. Thus, AVS could be very convenient for coarse grain integration such as manipulate and control the application flow of modular processes distributed in different hosts. When it comes to hundreds of modules connected by thousands of connections with high frequency event broadcast, what we need is an architecture that not only provides a high performance inter-modular communication, but also unleashes the power of well designed fine grained modules in the hands of a user. NeatTools fits into this kind of scenario in human computer interface very well and it provides flexible solutions as a general fine-grained visual-programming tool.

1. Summary of NeatTools Implementation
1. Design Goal of NeatTools

The design goal of NeatTools is to make it simple, object-oriented, network-ready, robust, secure, architecture neutral, portable, high-performance, multi-threaded, and dynamic.

1. Simple
1. Simple Usage for the Data-flow Network Designer

The goal of building a system that could be visually programmed without a lot of formal training in computer science was achieved. At the SigGraph Conference 98', a very large booth in the SigKids area allowed children (around 10-12 years old) to freely use a full version of NeatTools interactively as an experimental tool. Most children could pick up the concepts of NeatTools in a very short time and started exploring on their own. One of them was definitely the best of the best. He picked up all the major ideas in a half-hour and started to design his own data-flow networks ranging from a "metal detector" to "mouse position finder." This proves that the NeatTools visual programming environment design is naturally simple.

I added several concepts to NeatTools, which are based on purely visual programming aspect instead of the traditional textual programming. For example, most visual programming languages like AVS/Express and LabView, have distinct modes for data-flow network editing and execution, which are quite naturally accepted in traditional textual programming. Because every textual program source code needs to be edited and compiled before it can execute. Also, from the design point of view, making editing and execution mode exclusive will make the system design task easier. The mixture between editing and execution mode will cause a lot of technical difficulties especially in a multi-threaded system. NeatTools provides non-stop execution. So users can layout, execute, and debug data-flow networks at the same time without being interrupted.

2. Simple Programming Paradigm for the Module Designer

In module design, a programmer will first identify the module object, which will be used by the data-flow network design as the basic building block. Usually, this portion is most difficult because it will require broad knowledge of object-oriented programming and visual programming environment design. After the module object has been identified, the programmer could proceed to create a class which derived from the module base class or other module (when needed to inherit the common behavior). In terms of the process, creating a NeatTools module is very simple and straightforward.

2. Object-oriented

Object-oriented design is very powerful because it facilitates clean definition of interfaces and makes it possible to provide reusable software ICs. By software ICs, I mean the design hiding the implementation and information and only providing the interface as the communication media to the outside world. So the designer could focus on the design and implementation of software ICs without additional concerns.

In NeatTools, each module was designed as a special object unit. The object unit defines the module's visual representation, connection channel, connectivity of each channel, attribute, etc. All the interactions between modules are handled by the module's base class and NeatTools system. So the designer can focus on each module as an independent unit and may improve the algorithms or debug inside the module.

3. Network-ready

NeatTools integrates an extensive library of TCP/IP protocol routines into the visual programming building blocks. For examples the SocketObj and ServerSocketObj modules can be dropped into the desktop and create a network connection by just making connections which is much easier than in a traditional textual programming environment. In the future, to complete the network functionality, we will add more modules that can handle HTTP, FTP, and other network related protocols.

4. Robust

Most dynamic languages like LISP, TCL and Smalltalk are often used for prototyping. One of the reasons for their success at this is that they are very robust: we don't have to worry about freeing or corrupting memory. In Java, the programmer can be relatively fearless about dealing with memory because they don't have to worry about it getting corrupted. In C/C++, the pointer and memory management are the most problematic issues. So how could we make the NeatTools become robust and still use C/C++? I believe proper usage of object oriented programming is the key issue to achieve this goal. By hiding the pointer and memory management inside the data members, constructor, destructor, and assignment operator methods; the interface of object class could be free of pointer and eliminate the need of complex memory management -- it becomes distributed into each object class implementation.

 

5. Secure

NeatTools is designed to be used in network/distributed environments. By using the Socket/ServerSocket modules, NeatTools running on different computers could collaborate with each other through data exchange and events. Recently, a lot of emphasis has been placed on security. To enable a secure visual-programming environment, the addition of authentication modules that are based on public-key encryption was planned, users could use the authentication module to encrypt or decrypt events and close the door on most tampering activities.

6. Architecture Neutral

In the present computer market, application writers have to produce versions of their applications that are compatible with the IBM PC [36], Apple Macintosh [37], and other high-end workstations. With the PC market (through Windows/NT) diversifying into many CPU architectures, Apple moving off the 680x0 toward the PowerPC [37], and all the different brand named workstations based on different CPU architectures, production of software that runs on all platforms becomes nearly impossible.

In Java, the same version of the application runs on all platforms. But the Java bytecode relies on the Java runtime that allows the Java virtual machine to run the byte code. As demands come from all the different industries, Java runtime is constantly revised and becomes bigger and bigger. Because of that the browsers that carry Java virtual machine becomes huge, and they waste a lot of space when different Java runtime instances are needed by different browsers or Java development tool kits.

Some existing solutions, like the Win/U from Bristol Technology [38], provide biased implementations that favor a particular platform. For example, a lot of Microsoft Windows applications were written based on the Microsoft Foundation Class (MFC) [8], which is built on top of Win32 API. Win/U provides a layer that simulates the Win32 layer on the UNIX platform. So applications based on MFC could be compiled and run on the UNIX platform. This solution is based on the fact that lots of existing applications are already written for the Microsoft Windows platform. But it introduced unnecessary layer into the UNIX platform and decreases the application's performance.

NeatTools is built upon a Java Like Cross Platform API. In this API, it defines the basic language, networking, input/output, utility, and abstract window classes. It hides all platform dependent implementation details in the classes, provides a unified class interface to the applications that are based on this API and does not favor any of the platforms. By using this architecture neutral API, we only have one very thin layer between the application and the operating system it is running on. It guarantees that the application becomes cross-platform and will also have high performance.

7. Portable

Being architecture neutral is a big chunk of being portable, but there's more to it than that. All the platform dependent details are hiding in the object implementation. What the object interface provides is unified to all platforms. This is the classic usage of the object-oriented feature to build a clean portability boundary between the C/C++ POSIX subset, operating system, and applications.

8. High-performance

The architecture neutral, portable, Java like cross-platform API did provide the thin layer that ensured the high-performance of the NeatTools that built upon it. NeatTools' visual programming environment design is based on the module abstract model which simplified the object interaction into input and output actions. To provide the highest performance, NeatTools uses distributed messages broadcasts and direct function calls to implement the event broadcasts between modules. Unlike AVS, which uses the AVS kernel to provide centralized control over inter-modular communication and uses IPC data-flow protocol's RPC call to implement the channel event broadcast and data exchange, NeatTools has much superior performance over AVS on event broadcasting. According to the benchmark, NeatTools is capable of broadcasting 1,800,000 events in one second, which is not even possible for an AVS application.

9. Multi-threaded

There are many things going on at the same time in the world around us. Multithreading is a way of building applications with multiple threads. Unfortunately, writing programs that deal with many things happening at once can be much more difficult than writing in the conventional single-threaded C and C++ style. By using the Java like cross platform API, a sophisticated set of utility classes that provide the thread operations and synchronization primitives is provided. In Microsoft windows, the underlying implementation is based on Win32 Thread API. In UNIX, the underlying implementation is based on the MIT POSIX Thread software package.

In NeatTools, the interactive responsiveness and real-time behavior is very strong. All threads inside modules will not be interrupted or suspended by user operations like layout modules or making connections between modules. Everything is working concurrently and smoothly when the user is trying to group, resize, or move a set of modules. The multi-threaded feature in NeatTools did increase the productivity, functionality and usability.

10. Dynamic

To adapt to an evolving environment, NeatTools was designed to be extensible. External modules could be added into the system at runtime, which could extend the system, as new modules are available. In NeatTools, each module is a distinct object class, which could be put in the system statically as a build-in module or could be linked into the Dynamic Linkage Library (DLL) and, later on, loaded by the NeatTools system as an external module. The Polymorphism feature in Object Oriented Programming makes it possible for NeatTools to be extensible. The virtual function provides a way that enables a proper method implementation pointed to by a base class's virtual function table. So when a module is been dynamically linked, we could create an instance of the module and later on express its behavior through the common virtual functions inside the module base class.

2. NeatTools' Visual Programming Features
1. Introduction

Developing an application in NeatTools is very easy. Here is an example that shows how you could use mouse clicks to light up an LED. First, launch NeatTools, and set the desktop to "Edit" mode by pressing on the "ED" button in the left column. After opening the "Display" toolbox by pressing on the "DS" button, you are ready to start programming.

Now, move your mouse over to the button module and drag it to desktop. Then, move the mouse over the LED module and drag it to desktop also. Lastly, move the mouse over the output area of the button and drag it to the input area of the LED module. You will see a new connection built between the button and LED modules. Your first NeatTools program is ready and running. If you click on the button module, you will see the LED module light up. (Like in the following picture):



 

You can save your program by press the "SF" button and assign it to a file with an "NTL" extension. You always could load your program by pressing the "OF" button and selecting your NTL file to bring it back.

2. Multi-Thread Features

A NeatTools module can initiate its own threads depending on its design. A NeatTools user could just use it without notice. For example, you could put Time and Date Display modules on the desktop, and connect them. Now, you will see the Data module is showing the current time and updates every second. Then, you put the Timer, ClockDivider, and LEDs to the desktop, and connect them. You see the LEDs light up and cycle through each other. Right now, two threads are running at the same time inside the desktop area. By sharing the events, modules may share the same thread. For example, the four LEDs and ClockDivider share the same thread inside the Timer module.



 

3. Keyboard and Mouse Event Simulator/Filter

NeatTools could receive Keyboard or Mouse events through Keyboard or Mouse modules' outputs. In the following example, we use the keyboard "1" to control the LED and use 1DViewer to display the mouse X position.



 

NeatTools could generate Keyboard or Mouse events through the Keyboard or Mouse modules' inputs. The computer will "think" that someone is typing the keyboard or moving the mouse, but actually those events are generated by NeatTools. The following example uses mouse X position and 1DViewer to fire the keyboard module when mouse X position is in the right area. If a user launches an editor like Notepad or Word and puts it in the foreground, you will see a "1" character appear when you move the mouse from left to right.



 

This function is very important and useful. For example, if we want to help a disabled person to use a sensor to surf the Web, we can use NeatTools to pick up the signal and process it into the mouse or keyboard simulated events. Or, if we want to make NeatTools work with an existing application and can not change the application itself, we can program NeatTools to generate keyboard or mouse events to use and control the application. For fast prototyping purposes, this function is extremely useful because we always could connect and control an external application in no time.

4. Networking and TCP/IP

Transferring information to remote computers is very easy, after you layout the socket server and socket client, set the IP address, connection port, and enable both of them; you already have built an invisible duplex connection between these two modules. The socket and server socket pair can be put into different NeatTools processes or in a remote computer as long as the IP address is correct. With these Networking modules, a user can transmit any NeatTools data type to any computer running NeatTools. The data could be an integer, floating point, bytes, wave, or video data. Laying out an Internet phone using NeatTools is very easy also.



 

5. Container Nest Structure (Complex Module)

NeatTools could do visual modular design by hiding data flow networks inside a container module and becoming a complex module. Inside the container module, you can put in complex modules of other data flow networks. So a user could design an application using several layers. Each layer will handle different levels of complexity. This feature is rarely seen in most visual programming languages.



 

6. Transfer Focus among Text Fields

How to provide a full functional graphical user interface visually is always an interesting topic. For example, a text field module is needed, which could let a user type in characters and compose them into a character string. Usually, most applications will have several text fields for a user to type in. How do we provide a mechanism, which enables a user to transfer focus among text fields? Using mouse clicks to set the current focus would be an obvious way to do it. But for the user's convenience, some short cuts for the user to toggle or cycle through all the text fields or GUI components need to be provided. To implement the short cut feature, I added an input port called editFocus, which could trigger and set to the current focus by other modules. I also added an output port called nextFocus, which will send out a true event when it is in focus and the user presses the TAB key. By connecting the nextFocus output port into the next text field's editFocus input port, we could build a form with several text fields and a user could use the TAB key to cycle through the fields.



 

7. Polymorph Data Type

NeatTools is a strong typed visual programming language. Usually, it is desirable to connect input and output ends only when they are of the same data type. The NeatTools system guards that very well, but sometimes, it is not so convenient to implement all the different modules for each different data type. The ControlObj module is one of the modules that needs the polymorph data type. ControlObj module works as a switch controlled by enabling input signals on the top, which is very useful for all kinds of data. With polymorph data types, the ControlObj input/output pairs could connect to any data type. For example, after connecting the data input port to an integer port, the input/output pair adapts itself into the integer data type. Then the corresponding output port could accept integer data. With the polymorph data type, we make a balance between strong typed and weak typed checking and make NeatTools more convenient and efficient. In the following example, I use ControlObj to control integer, date, and real number at the same time.



 

8. Multimedia Features

NeatTools has added most of the multimedia features into modules including Mixer, Wave input/output devices and MIDI input/output devices. Now, a signal that comes in from any device or remote computer can control the CD volume, play a MIDI song, hear sound wave, etc. The information perception could be anything we ever think of; we could "hear" the image, or "see" the sound. We plan to add more modules into NeatTools to handle graphics visualization and data processes. In the information age what we need is not just information, we need all kinds of different ways to observe information and abstract the patterns and features we want.

The following is an example in which NeatTools is used to visualize sound wave intensity.



 

9. Multimedia Database

Most people consider a database as belonging to a textual programming API layer, but I personally think providing a visual database is a very interesting concept. Different databases could use connections to build the relations between databases and become a visual relational database network. The relations between databases could be indexing, key word match, or condition match. The indexing relation is quite straightforward, users just connect an integer field value to the index input of another database. Whenever the integer field value changes, it will locate the connected database into the corresponding record. The keyword match and condition match relations work likewise.

In the following example, the Session database file contains several fields including patient's SSN and therapist's Reference Number. Session data uses the SSN field to build a relation with the Patient database by connecting from its SSN output to the Patient database's search of SSN input. Session data uses Therapist_Ref field to build a relation with the Therapist database by connecting from its Therapist_Ref output to Therapist database's search on Ref_No input. Therefore, when we navigate the Session database by changing its index, the Patient and Therapist_Ref database will move to the correspond record.



 

10. External Module and Dynamic Link Library

In NeatTools, modules can be built-in or external. The external module will be a dynamic link library. NeatTools could load the external module in run time. The external module could be compiled separately, so NeatTools becomes scalable. Users could expand NeatTools' functionality by adding their own modules. Also, some device manufacturers provide their own API in DLL. If we make an external module DLL that includes the manufacturer's API, we could reduce the NeatTools system's dependency on the manufacturer's DLL and keep the NeatTools system core compact and small.

The following picture shows the dialog box that NeatTools uses to load the external modules.



 

11. State Machine

In many control-based applications, the state diagram is the dominant aspect of the model that can capture the dynamic behavior. The major concepts of dynamic modeling are events, which are the stimuli, states and configurations of object behaviors. In NeatTools, events are conducted by connections between modules, and states are represented by a special State module which can hold a single named state and can stimulate the connected state modules when conditions are matched. A State module can have several outgoing connections to other state modules. Each connection can only connect to one state module and can be stimulated by one condition pulse input event.

The following is a very simple example of a state machine. There are 3 states, A, B, and C. Each state could be triggered by a condition and stimulate the next connected state module. A state module could be setup to trigger by several conditions and connect to several other state modules. I will put a detailed example in next section.



 

3. NeatTools Architecture

The NeatTools architecture defines the complete structure for implementing NeatTools' cross-platform, extensive module unit, usability, and functionality features by using UML (Unified Modeling Language).

1. Three-Layer Architecture

The NeatTools' implementation model is defined as a three-layer architecture. This architecture is necessary for cross-platform design. There are several advantages associated with this approach:

1. It separates the application design from platform dependent details by providing abstract windows and a system resource layer for building the application upon the abstract layer.
2. Based upon a well defined interface in the class, a system programmer can refine the design and algorithms later without needing to change the dependent layer.
3. Current commercial window abstract layers, like Microsoft MFC, Borland OWL, and OSF MOTIF tend to be fat in design and fixed on one specific platform.

The conceptual framework for NeatTools is based on architecture with three layers:

1. Operating system and programming language layer
2. Java like cross-platform API layer
3. NeatTools application layer

Functions of the layers are summarized in the following table:

Table 1: Summary of layers

The operating system and programming language layer provides all the basic system services and resources. This layer includes Win32 API, X11 API, and C++ run time library. The Win32 API and X11 API provide the generally required functionality of windowing, graphics context, objects and memory management in a platform dependent fashion. The C++ run time usually comes with the actual compiler implementations. There are even some standards proposed for the C++ language itself. However, there are still some significant differences between companies and platforms. Hence, it is desirable to have an abstract layer, which provides standardized interfacing between the application layer and the operating system layer.

The Java-like cross platform API layer is the abstract layer that works between the operating system and the application. It releases the application's dependency on the operating system and provides a high-level utility class object support. Usually, the API is a collection of procedures. It is usually very difficult for programmers to find out the relevant procedures in an API; know its capability and usage; know how it interacts with other procedures; not to mention putting them together in order to accomplish a required task. In this sense, a cross-platform API layer is necessary and desirable.

Now comes the second question: why Java-like. "Java-like" means it defines the class and interface according to a Java API specification. NeatTools needs to be fast and efficient to handle real-time and computationally intensive tasks. By implementing a Java-like API in C++, we gain Java’s ability to run on different platforms as well as the high performance of C++. The application layer is where we build the actual NeatTools implementation in cross platform fashion. Different applications will eventually need some special classes which could perform the usability and functionality needed for the particular applications. So the presence of this layer is natural.

1. Package Structure

The NeatTools implementation is moderately complex. It is composed of approximately 100 classes in a Java-like cross platform API layer and approximately 120 classes in a NeatTools application layer. The complexity of the classes is managed by organizing them into logical packages. The NeatTools implementation is decomposed into the top-level packages shown in Figure 1:



Figure 1: Top-level Packages

2. OS and C++ Runtime Package

The Operating system and C++ runtime package is further decomposed as shown in Figure 2:



Figure 2: OS and C++ Runtime Layer

3. Java-like API package

The Java-like cross platform package is further decomposed into NET, AWT, UTIL, IO, and LANG packages as shown in Figure 3:

Figure 3: Java like Cross-Platform API Layer

The exception classes structure inside the Java-like cross platform packages is illustrated as shown in Figure 4:

Figure 4: Exception Class Diagram in Java Like API Layer

1. Experiments
1. Case Descriptions
1. Eyal Sherman

When he was 5, Eyal Sherman suffered from a stroke. After surgery, Eyal was a quadriplegic, with severely limited movement. Communication was restricted to yes and no questions or lip-reading. It had been 10 years since this tragedy, and life had settled into a kind of norm for the Shermans. Eyal attends public schools, where he is enrolled in New York State Regents classes, a program for advanced students.

2. Brooke Kendrick

Brooke is a delightful 7 years old who is diagnosed with spastic quadriplegia due to cerebral palsy. She functions at a pedal gross motor level. She uses a power wheelchair with a joystick control, which she is presently learning to use. Brooke is non-verbal but does make her desires known by smiling, laughing, crying and saying yes and no. She is a social little girl who appears well aware of what is going on around her. She continues to demonstrate a strong startle reflex when loud or unexpected noise is heard. Brooke attends an integrated First Grade program at Lakeland Elementary School. This is a full day program, which integrates her educational and therapeutic activities. She has a one-on-one teaching assistant.

2. Approach

The NeatTools project team has been working with Eyal Sherman closely since 1996. The approach was divided into hardware and software portions. The hardware team developed the EMG (electromyograph) detector known as TNG and did experiments with all kinds of headsets and mounting systems for Eyal. They also tried a lot of sensor technologies including photocells, displacement potentiometers, Hall Effect transducers (magnetic sensors), pressure transducers, bend sensors, etc. Jo Johanson was the one who mainly developed the Neat DOS. Later on, he briefly experimented on a Windows and Java version. TNG devices pick up the signal from the sensors on the headset and transmit them into the computer. The Neat software keeps receiving the signals, processing them, and later on maps the signals onto actions of external devices.

3. Limitations of Early Software

The old design, especially the DOS version, is a great tool to receive signals from external devices, change the configuration dynamically by making connections, and then do logic operations on the buttons. The major draw back was that modules have a somewhat limited layout. The format is fixed and could not provide the full flexibility for user to layout. For example, a user could add new buttons. After button creation, a user would see a new button appear at the end of the button area. A user could add connections to a button, but could not move the button or resize the button itself. Also DOS does not support multi-process, multi-thread, TCP/IP, and multi-media, as there are many limitations inherent to the DOS operating system.

The Java version improved the layout mechanism. Users could create new modules and put them anywhere on the desktop. This version uses a native method to implement the COM port related function and runs under Java enabled browsers. But, as the JDK changed, the new browsers prevented Native methods from low level COM port access for security reasons. Without COM port access, the Neat software could not read the signals from the TNG device and so would sit in the dark. Other than the COM access problem, there is no simple way of communication with other existing applications. For example, to enable the user to employ a TNG signal to control a Java Tetris game, the software team member has to find out the source code, download it and rewrite the code. Thus, Neat software could create and control the game directly, but when it comes to commercial software applications like Netscape and Word, there is no way we could acquire the source code and try to modify a huge application and combine it into Neat software. This approach is not feasible at all.

4. Ideas and improvements

At this point, the group tried to find a Java programmer to solve the problems. On one evening, when Eyal had a session with the NeatTools project team, I came by to visit them and tried to understand the nature of the project and the problems they were facing.

1. The mouse driver prototype

The first thing I noticed is that to try to modify the source code of applications in order to control them is not realistic. There are hundreds and thousands of applications out there. How could we modify them all? Also, most commercial applications will not release their source code. Then how do we provide a way for disabled persons like Eyal to use them? Secondly, Java is changing so fast, and its future is still not certain yet. Right now, the new browsers prohibit us from COM port access in the native method, and JDK itself does not provide the functionality to access the COM ports, (and will not in the future, because it is meant to be a cross platform language). Why should we limit ourselves by using Java? I happened to have the knowledge of how to simulate mouse events and how to install a mouse event filter in the Windows environment, so I decided to give it a try.

Three days later, I came out with a very small C windows program (executable around 60k). It has some simple controls and property settings on its front panel. After it connects to the COM port, the signal from TNG will come in and display in its client area. By changing the signal, it could generate mouse events, change mouse position, and simulate mouse click. When we put this small program into the background and brought other applications to the foreground, we were able to use the signal from a TNG to control any application without having to rewrite it!

This prototype mouse program solved some of the major problems, but it was still not good enough to generally adapt to changing situations. For example, I hard coded the mapping between a user gesture signal and the mouse events. But, if a patient could not do some of the predefined gesture actions and needed to change it into other gesture actions, then I would have to expand the program to make the random gesture actions/mouse events mapping possible. Also, TNG is just one of the input devices, there are many different devices out there, thus how could we provide a generic way for a user to configure all of them easily?

2. Next Generation Neat Software - NeatTools

To make it adapt to changing situations, the data flow network with a visual programming environment is the way to go. This time, I wanted to make it general and powerful. I devote the next section to discussing the key issues in building a better Neat software.

5. Key Issues in Building Better Neat Software
1. Java Like Cross Platform API in C++

The Java Like Cross Platform API in C++ is actually a very thin layer, which is provided as an interface to the operating system, C++ runtime library, and Windows system. This API hides the platform dependent implementation and unhooks the application's dependency from the operating system and Windows system.

Other than following the JDK standard, the challenge was actually in how to map the Windows handle, related operations and Window messages into the object that represent the window object. It has to provide an efficient way to associate the window events into object's methods and a unified way to map the same object into window objects of different windows system like Microsoft Win95/NT and UNIX X Windows.

Usually, programmers of Microsoft Windows will use the MFC to ease the complex process of registering Windows class, setup parameters, create windows, and finally display it and start to process window events. By using MFC on UNIX, a layer must be provided to simulate Win32 API on top of X Lib (and MOTIF sometimes). The Win/U product by Bristol Technology was one of the products that provided the cross platform solutions on UNIX. But this solution has a bias on UNIX platforms, because an application will have two layers between itself and X Lib, which needless to say will decrease the application's performance on UNIX platforms. So, to be fair to both major Windows operating systems, Java-like Cross Platform API in C++ is the key concept for NeatTools to become a cross platform and high performance application.

2. Module Coordinate System

Most visual programming environments use integers to represent the current position and dimension. They can not scale the desktop, or scale only in limited settings. Eventually, when data flow networks become complex, a user could use up all the space and have to overlap the modules and connections. I proposed using a floating point instead. This change enabled the desktop to have arbitrary scale. A user could have a layers design, where each layer has different visibility and could handle the complexity locally. A module will know its current screen dimension and position in integer pixel units only when its repaint or update method are invoked. NeatTools not only supports a scaling system that could scale a design to arbitrary scale, but also provides rulers and scroll bars to help users locate or reposition a design without the limitations of screen resolution.

3. Module Event Broadcast Model

As I discussed in the previous section about NeatTools' implementing concept, the most important key to a high performance fine grain visual-programming environment is how the event broadcast performance is increased.

In NeatTools, the module abstraction ensures that there is almost no dependency of one module on another, except for the data objects that flow in and out of a module, software modularity and reusability is enforced and highly preserved in the system. The same idea was presented in most visual programming environments, but different implementation architectures could lead to dramatic performance differences based on the same idea. For example, in AVS, the AVS kernel provides centralized control over the inter-modular communication. Restricted by the AVS kernel and its IPC data-flow protocol, any communication between two parallel modules has to go through the sequential host-node channel and the module-module pipeline on the control processor. In this sense, the current AVS kernel serialized parallel channels. This could become a performance bottleneck. Not to mention the IPC data-flow protocol's RPC call is much slower than a direct function call. AVS kernel design has its basis in the distributed nature of AVS modules. But this design limited AVS as a coarse grain visual programming tool. All the fine-grained operations and functionality have to be implemented inside the module and can not be exposed and utilized by the user directly through the visual programming environment.

In contrast to AVS kernel's centralized broadcast control model, NeatTools not only has a centralized storage for all logical connections but also every NeatTools module has a local reference to its logical connections (including input and output connections). When broadcasting events, NeatTools modules can make direct access to logical connection information and invoke the destination module's engine method directly. Different modules could issue parallel events under different threads. This is the distributed broadcast model in NeatTools and proves it could meet the high performance requirements of a fine grain visual-programming environment.

4. Increase Usability

In most visual programming systems including LabView and AVS/Express, when in edit mode, a user could put in new modules, make connections, copy, or delete modules, etc. When editing is finished, the user will have to press a button or select an option to tell the system to enter the execution mode, observe the execution and, later on, stop it and resume edit mode for the next development cycle. When the program is in execution mode, a user can not change the layout, connections, or modules. From a programmer's point of view, it is always technically easier to separate the operations into edit and execution mode exclusively. Thus, the implementation could concentrate on a single scenario without having to worry about the interactions between different operation modes. Yet, from the user's point of view, somehow this kind of distinction increases the time of development. It is quite often that the user could make some changes in edit mode and start the execution mode. Then he observes the problems or bugs and could not modify the network right away. He has to stop the execution and start the edit mode again. Just like traditional textual programming, the user has to use the editor to modify the source code, compile it, run it, and eventually gets back to the editor to modify again. I personally think that a good visual programming environment should not make edit and execution modes exclusive. Instead, the designer should use the user's prospect and try to make it truly user friendly within the time and space constraints.

NeatTools provides a non-stop execute mode, and users can turn the edit mode on and off as they wish. This is actually a difficult task even for an experienced programmer, because, each module could start its thread and the following concurrent situations could happen: a user tries to delete a connection but the source module is broadcasting messages through this connection. Or, a user tries to delete a module, but this module is still updating itself through graphics context. It is usually very hard to find bugs inside a multi-threading system, because the bug will not always occur. There are always possibilities for it to occur, but you never know exactly when it will. This makes debugging in multi-threading systems sometimes almost impossible. Thus, how we implement the functionality and make it thread-safe is a most important topic. In NeatTools, the technique I use is to first identify the critical sections inside operation implementation sections. Critical sections are usually operations involving those resources shared by more than one thread. Second, I carefully analyze the critical sections and add exclusive mutex only when it is necessary. Theoretically, this process looks simple, but to identify critical sections, bring out a thread-safe design, and still provide high performance; one needs knowledge and experience in multi-threading systems and concurrency issues.

5. Identify the Modules

How we identify the module as a functional and reusable object unit is the most important and complicated task. If we liken the NeatTools system to the human body, then the modules are the different organs that work together.

For example, in the previous section, the mouse driver prototype was a small program that performed data acquisition, process, and action mapping. In NeatTools, the desktop provides a visual-programming environment where users can create, manipulate modules, and make connections. We still have to identify and design the module in order to restore the functionality provided in the mouse driver. In the mouse driver's case, we could have COM port, TNG, Calibrator, Viewer, Mouse Simulator, and some logical operation unit to provide the gesture recognition. The module could be a unique entity in the computer like COM ports that represent the physical hardware ports and the software functions related operations like open/close the COM and input/output byte stream, etc. It could be a protocol, like the TNG that receives the byte stream from COM and implements the TNG protocol that extracts the signal information from the byte stream. Or, it could be a calculation unit that provides computational services, like AND, OR, XOR, Calibrator, etc. Identifying modules is important because the requirement specifications and design will use them as the building blocks. Mistakes in properly identifying them will affect the extensibility and maintainability of the data flow network.

6. Dynamic Ports

Most visual programming environments do not provide dynamic ports for their modules. By dynamic ports, I mean that the input and output ports of a module could be changed in run time. For example, a database module could have several input/output ports for its fields in a database. When a database module is assigned to a different database file, its port configurations (like number, data type, and tags) change dynamically.

7. Display Feedback Mechanism

NeatTools is meant to be a tool that can apply in HCI (Human Computer Interface) and the tele-medicine field. The display feedback lag time should be minimized when possible. Usually, the display procedure inside a module will involve operations to Windows graphics contexts, which are relatively slow compared to most other non-display operations. In some situations, especially in fine-grained data flow network design, high frequency events may trigger a sequence of display refresh in modules. Sometimes, it could slow down the system a great deal or even hang the system completely.

There are two approaches that could be combined to provide a better solution. First, decrease the overhead of module display. In NeatTools, modules could be resized or overlapped by users. When a module requests a display refresh to itself, a graphics context has to be created and a region has to calculated before the module's update method could be invoked. Region is a logical set of pixels or rectangles that could associate with a graphics context to determine the region that could apply or prohibit the graphics operations. Region can be a simple rectangle or very complex shape because of overlap by several other modules of different dimensions. It is an O(n) operation for a desktop with n modules to calculate the logical region that modules associate with a graphics context. This could be a very expensive operation, especially when the number of modules in a data flow network becomes large. It becomes even worse when a module needs to update itself frequently. To eliminate the region calculation associated with each module display update, I add a region data member inside the module and make the region calculation when it needs to be done like create, move, duplicate, or resize the modules. This change makes the module display refresh an O(1) operation with much less overhead. Second, some repeated display refresh needs to be eliminated when possible. In modern windows systems, there are usually some mechanisms to decrease the need of window redraw. In Microsoft Windows, when an application sends out several invalidation requests to the window system, it will combine the invalidate regions into a bigger region and send out less paint requests back to the application. In X Windows it has the similar mechanism. But, this is not good enough for NeatTools. Because a paint request from the Windows system will cause the desktop to trigger an O(n) calculation to determine which modules need to be updated. To decrease the paint requests from windows system when a module updates itself, it calls a utility method that puts itself into a queue. Later, when the system is idle, a procedure will pick up the queue, delete the repeated request and refresh the modules only once. The module itself has to cooperate with this mechanism so that it should be able to make incremental updates without trouble from missing refresh requests. These two approaches proved to increase the module refresh and increase efficiency by utilizing the CPU idle cycle.

8. Cyclic Data Flow Network Issues

In a data flow network with synchronized event dispatching mechanisms, a cyclic data flow network could cause a system to hang by infinite loop. So, instead of avoiding cyclic networks, NeatTools allows a cyclic data flow network to exist without hanging itself. The speed is always the key concern in NeatTools. Thus, how we could propose an approach to detect the cyclic network efficiently and prevent the infinite loop was quite a headache.

In NeatTools, a module notifies the connected modules that an event needs to be processed by invoking the broadcast utility method, so it is true that when a module's broadcast method on a particular port is invoked recursively more than once, a cyclic situation is found. By using this observation, we could add a counter that associates with each output port and identify the cycle without extra analysis methods. But, when it comes to multi-threaded environments like NeatTools, this approach will not always work. For example, a module issues a broadcast event and at the same time another module uses a different thread and issues an event that flows through the same broadcast port. The counter that associates with the broadcast port will detect a cyclic situation but actually it isn't. In this case, adding a mutex inside the broadcast method would be enough to prevent the error in cycle detection. But adding the mutex as a default could decrease the parallelism in event broadcasting a great deal. And, depending on the module design, not every port would always have multi-threading events flowing through. Here, I leave it to the module designer's hand to determine where the critical sections are that need the mutex added to avoid concurrent problems. The users who design data flow networks just use the modules and make connections. They don't have to know what is going on under the hood.

9. Thread and Concurrent Related Issues

Using threads properly could increase the system utilization by eliminating unnecessary busy waiting loops. For example, in the receiving portion of a Socket module, we never know when the next event will come in through the network. The traditional way is to use a busy waiting loop and non-blocking sockets to keep reading on the socket or use the select function to check the socket status. These loops will usually cause busy waiting and waste the CPU cycle. By initiating a thread and using a regular blocking socket to read the socket in the thread, when no data comes in, the socket will block the thread. The application's main thread is still working without having been blocked by the socket read operation. That way, we could maximize the performance without complex implementation and time sharing inside the application and leave the problems to the operating system that provided the threads.

In NeatTools, each module is allowed to have its own threads, yet it depends on the module designer's decisions. But there are some issues that have to be taken care of. For example, deleting a module without destroying the associated threads could lead to a system crash, waste resources, or cause a memory leak. Destroying a working thread without "join thread" operation could cause an event broadcast to be interrupted abnormally and cause problems. So, in the appendix, I dedicate some discussion to focus on the guidelines and tips on using threading in the NeatTools module efficiently and correctly. But, even if module designers follow the guidelines to create, run, join, and destroy the thread properly, deadlock still could happen in some special situations. For example, a module initiates and starts a thread. In this thread, it broadcasts events in a loop. Eventually, one of the events caused the module itself to close. Now the problem comes, when the module's close method is invoked, it will try to use the "waitFor" method to join the thread so the loop inside the thread will have a completed cycle and exit successfully. In this case, the "waitFor" method would never return because the loop is stuck in the event broadcast process, which is stuck in the "waitFor" method itself. Now deadlock happens and the system hangs forever. To prevent this particular situation, a user could use mutex and some extra counter variable to keep track of the possible deadlock. My suggestion to the thread issue is, use it only when necessary and always use it with care.

6. Experiments

During the NeatTools' construction stage, I kept improving the functionality by putting in more modules and increasing the usability by providing more services in the desktop area. I kept a working page at http://www.pulsar.org/ej/work.html and put in descriptions and experimental data flow networks with screen dumps. Thus, in parallel to the hardware team's effort on the headset, improving sensor technology, and TNG device, I cooperated with the hardware team by team work on developing a new TNG protocol, new modules, and sample data flow networks for them to learn and design more applications for Eyal and Brooke.

After I finished the basic functionality of the mouse driver prototype level, I laid out a network for experimental purposes. At this point, NeatTools could do exactly what the old mouse driver prototype could do, and we could start to make all kinds of different network designs to improve it and make it more useful, not just a fixed, hard coded program.



 

Later on, I proposed more prototype networks for Eyal to control the mouse. This time, I used signal gesture to control the mouse direction, movement and mouse button click.



 

I laid out another network that would let Eyal control the highlighted column and row of a keyboard module and use it to type characters into other applications like Word, Notepad, etc. By using this network, Eyal was able to type characters into a computer for the first time in his life. Later on, we added an assistant program that could read the words he typed.



 

We keep improving the software and hardware. For example, the hardware team used photocells a great deal. They are cheap and sensitive to light and very good at picking up the facial expressions signals. But photocells change their signal level when an environment's brightness changes. To accommodate this situation, the team member in the session has to keep calibrating once in a while. Sometimes, it is too much trouble for the family member who is not quite familiar with computer operations. So, the hardware team keeps trying to find something that has a steadier signal output. They tried micro switches and, eventually, came across a special joystick that has a soft rubber at the end of the stick. Eyal could use his mouth to control the joystick very well. Around the same time, Dr. Edward Lipson laid out a very complex and sophisticated data flow network in NeatTools called Joy-Mouse.



 

In the Joy-Mouse data flow network, a user could precisely position the mouse cursor under the control of a joystick. This can be either a standard computer joystick, or a custom one (e.g. small chin joystick for Eyal) that delivers XY signals to analog inputs of a computer interface module such as TNG-3. The adjustable nonlinear profiles in the quadratic and cubic modes provide precise control of the cursor for small joystick deflections, while allowing rapid movement for large deflections. For more information please refer to the URL: http://www.pulsar.org/neattools/edl/joymouse_docs/JoyMouseManual.html

Dr. Edward Lipson happened to come across a commercial product called FITALY. The small application provides a keyboard interface. A user could use mouse clicks to generate virtually any key stokes. The most important aspect is that the keyboard interface is arranged according to the character frequency analyzed by huge sets of documents, so, it minimizes the mouse movements when trying to use FITALY to generate key stokes.



 

7. Results

By using Joy-Mouse, NeatTools, FITALY keyboard, TNG-3 device, Headsets, and sensors, we started to provide a feasible way for Eyal to surf the web, use applications, and send e-mail to his friends. For Brooke, the hardware team designed all kinds of foot pedals and hand devices. Combined with Joy-Mouse, NeatTools, and TNG-3 device, she could now use her hands and feet to control commercial educational applications, learn from the courses inside applications, and have her progress evaluated.

8. Conclusion

In my experiments, NeatTools did provide a powerful and affordable way to improve the life of disabled people like Eyal and Brooke. But, there is always room to improve our approaches. For example, using facial express to type characters by using Joy-Mouse and the FITALY keyboard is still very slow compared to normal people typing by hand. How could we provide a more sophisticated method to map customized facial expressions directly into characters, words, or event sentences? How we could add more AI support in helping them self correct and increase the human computer interface throughput? These are great ideas for future development.

2. Conclusion
1. Future Development

I have worked on NeatTools for around 18 months. Right now, I have already finished most of the core structure and functionality. Scaleable software development like NeatTools usually never comes to an end. There is always plenty of room to make some adjustments and improvements.

1. Aggregate Data Type

The Aggregate data type is an interesting concept. NeatTools already provides a way to extend new data types and register them into the system. If a user could aggregate different data types into a single complex data type, later on, provided the mechanism to construct nested complex data types visually, it would be very useful for some applications needed to handle several data members at the same time. For example, if we could provide aggregate data types that have X, Y, and Z data members inside and use it as a single 3D point data type, it would increase the readability and efficiency of the data flow network.

2. Connection Visibility

When data flow networks become more and more complex over time, the connections and modules are all over the desktop area and become very hard to tell apart. Even worse, some connections may overlap partially or even completely, or some other modules may cover them. How we could provide a method to solve this situation did take a lot of effort. Currently, NeatTools has a highlight mechanism, which will highlight the current closest connection or module and display the related information in a status bar. Thus, users can perform some operations on the highlighted connection or module. But it still could not handle the overlap or covered connections very well. I wish I could have an efficient algorithm to arrange all the connections and minimize the overlap between connections, so the connections would always be very clear and easy to read.

3. Undo/Redo Features

Although undo is a common and rather simple feature in word processors and spreadsheets, undo in a graphical programming language is a very sophisticated and challenging utility. An extensive, versatile undo/redo feature in NeatTools will take a lot of time to implement. This feature is extremely important to the NeatTools user. I wish I were able to add this feature in the near future.

4. Network Support Modules

Currently, the Socket and ServerSocket modules already support the TCP/IP protocol. To enable NeatTools users with the full power of processing the vast information from the web, FTP sites, and news servers, we will develop WWW, FTP, and NEWS modules to provide the service and serve as information sources that could be processed and visualized by NeatTools.

5. Authentication Modules

To enable a secure visual-programming environment in a network/distributed environment, we will need authentication modules that facilitate public-key encryption to protect the network communication between NeatTools from tamper activities.

6. Visualization Modules

Visualization modules always play an important role in human computer interface tools. As a human computer interface visual-programming tool, NeatTools will need more visualization modules in the future. Currently, we have some standard 1D viewer, 2D viewer, sliders, LED, etc. The 3D viewer, Space-time viewer, and 3D surface visualization modules are yet to come. Those visualization modules will provide key functionality to facilitate future human computer interface activities.

7. Artistic Modules

Currently, I have designed the modules emphasizing on their functionality. So, sometimes, the appearance is not so appealing. In the future, we could consider adding more artistic modules in NeatTools. It could provide functionality as well as the overall look and feel. About the artistic aspect, we would need people with computer graphics training and also a good sense of artistic design. A lot of commercial applications including LabView do a very good job on this aspect. NeatTools needs to work on this more.

2. Conclusion

The focus of this thesis is my work on proposing a new architecture of fine-grained data flow network visual programming environment - NeatTools. I have detailed descriptions about module abstraction, which forms the module event broadcast model in NeatTools. I also identify and specify some of the related systems and ideas. During the experiments for Eyal and Brooke, several key issues about building a visual programming environment were explored and discussed.

Our major findings are:

1. The same idea could have variant performance based on different design concepts and implementation architectures. For example, both AVS and NeatTools are provided as data flow visual programming environments. The AVS kernel provided the centralized control over the inter-modular communication, and hence serialized parallel channels. On the contrary, in NeatTools, different modules could issue parallel events under different threads and become a distributed event broadcast system. So, NeatTools has higher performance in event broadcasting and could meet the requirements of a fine-grain data-flow visual-programming environment.
2. The coarse-grained data flow model is well suited for combining subsystems and providing integrative solutions. But, all the fine-grain operations and functionality have to implement inside the modules and can not be utilized by users directly through the visual programming environment. It could reduce the function reusability and increase the implement redundancy. In this sense, a fine-grained data flow model would be a better choice for a designer to identify the reusable objects as the building blocks.
3. Use of commercial application frameworks like MFC to construct windows applications seems to be obvious for managing the complexity of a low level API like Win32 API. But it could cause an application to become native to a specific platform and become very hard to port to other platform like X Windows. Without modifying the application itself, one could build a simulated Win32 layer on top of UNIX and X Lib (and MOTIF) to provide the cross platform solution (Like Win/U by Bristol Technology). But this solution could introduce extra layers and decrease the application's performance. If we could add a unifying thin layer between a Windows operating system and an application, cross platform and high performance could be achieved at the same time.
4. To increase the user development speed, distinct and exclusive modes like edit and execution are not desirable; it is necessary in traditional textual programming environment, because the source code needs to be compiled before it can be executed. In a visual programming environment, we could keep the data flow network in constant running mode without interrupts by edit mode operations. Also, we could maintain maximum performance and solve all possible concurrent problems. Then, users could design, execute, and debug at the same time and increase the development throughput.

1. Appendix
1. NeatTools Reference Manual
1. NeatTools Module Specification:



Name: NOTObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise NOT operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) NOT operation. When it is 16, it will become the 16 bits wise NOT operation.



Name: ANDObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise AND operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) AND operation. When it is 16, it will become the 16 bits wise AND operation.



Name: ORObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise OR operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) OR operation. When it is 16, it will become the 16 bits wise OR operation.



Name: XORObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the bit wise XOR operation on the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical (1-bit) XOR operation. When it is 16, it will become the 16 bits wise XOR operation.



Name: GreaterThanObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is larger "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: GreaterEqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is greater or equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: EqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: NotEqualObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is not equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: AddObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate the sum of all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: MultiplyObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Multiply all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: SubstractObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" - "input-1" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: DivideObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" / "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: AbsObj

Input: "input" integer port on the left with maximum one connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate ABS("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

 



Name: PowObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" ^ "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: MaxObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Choose the maximum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: MinObj

Input: "input" integer port on the left edge with unlimited connections.

Output: "output" integer port on the right edge with unlimited connections.

Function: Choose the minimum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RemainObj

Input: "input-1" integer port on the left edge and "input-2" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" % "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RandomObj

Input: "control(logical)" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will generate a random integer number and broadcast the result value to "output" port.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: ControlObj

Input: "N(1-512)" and "control(logical)" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorph ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" polymorphs port on the right edge with unlimited connections.

Function: "N(1-512)" will determine the number of input-output pair. When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will connect the input-output pairs. So when there are input events presented on say "in-k", this module will transmit them into the corresponding "out-k" output port.

Properties: "moduleColor" set the background color. "N(1-512)" set the number of the input-output pair.



Name: SampleObj

Input: "N(1-512)" and "control(logical)" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorph ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" polymorphs port on the right edge with unlimited connections.

Function: "N(1-512)" will determine the number of input-output pair. When detecting a raise edge signal event (from false to true) on the "control(logical)" port, this module will sample the current input value from input port say "in-k" and transmit into the corresponding "out-k" output port.

Properties: "moduleColor" set the background color. "N(1-512)" set the number of the input-output pair.



Name: PulseObj

Input: "input(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving an input event, this module will generate an true-false event pair to the "output" port.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: DelayObj

Input: "input(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving an input event, this module will hold the value and broadcast the value when next input event comes in.

Properties: "moduleColor" set the background color.



Name: AccumulatorObj

Input: "enable(logical)" integer port on the top edge and "clock(logical)" integer port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When receiving a raise signal (from false to true) event on "clock" port and current value of "enable" port is true, this module will increase the internal integer accumulator and broadcast the value to the "output" port. If receive a drop signal (from true to false) event on "enable" port, the internal integer accumulator will reset to zero and broadcast the value to the "output" port.

Properties: "moduleColor" set the background color.



Name: MultiplexerObj

Input: "N(2-512)" and "select" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" polymorph ports on the left edge with maximum one connection.

Output: "output" polymorphs port on the right edge with unlimited connections.

Function: "N(2-512)" will determine the number of input port. When receiving an input event on "select" port say k, this module will connect the selected "in-k" input port to "output" port. So when there are input events presented on "in-k" port, this module will transmit them into the "output" port.

Properties: "moduleColor" set the background color. "N(2-512)" set the number of the input port. "select (0-(N-1))" set the current selected input port.



Name: DeMultiplexerObj

Input: "N(2-512)" and "select" integer port on the top edge and "input" polymorph ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" polymorphs port on the right edge with unlimited connections.

Function: "N(2-512)" will determine the number of output port. When receiving an input event on "select" port say k, this module will connect the selected "input" input port to "out-k" port. So when there are input events presented on "input" port, this module will transmit them into the "out-k" port.

Properties: "moduleColor" set the background color. "N(2-512)" set the number of the output port. "select (0-(N-1))" set the current selected output port.



Name: EncoderObj

Input: "N(2-32)" integer port on the top edge and "in-0", "in-1" ... "in-(N-1)" integer ports on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: "N(2-32)" will determine the number of input port. When receiving an input event on input port, this module will encode all the output value, encode it into an integer and transmit them into the "output" port. For example, when N = 4 and the input value from "in-0" ... "in-3" are 1, 0, 0, 1, this module will encode 1001 binary value into output integer value 9.

Properties: "moduleColor" set the background color. "N(2-32)" set the number of the input port.



Name: DecoderObj

Input: "N(2-32)" integer port on the top edge and "input" integer ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" integer port on the right edge with unlimited connections.

Function: "N(2-32)" will determine the number of input port. When receiving an input event on input port, this module will decode the input value into the decoded bit value and transmit them into the corresponding output port. For example, when N = 4 and the input value is integer 9, this module will decode it into 1001 binary value and send out 1, 0, 0, and 1 into "out-0", "out-1", "out-2", and "out-3" output ports.

Properties: "moduleColor" set the background color. "N(2-32)" set the number of the output port.



Name: ClockDividerObj

Input: "N(2-512)" integer port on the top edge and "clock(logical)" integer ports on the left edge with maximum one connection.

Output: "out-0", "out-1" ... "out-(N-1)" integer port on the right edge and "value" integer port on the bottom edge with unlimited connections.

Function: "N(2-512)" will determine the number of output port. When receiving an raise signal (from false to true) on "clock(logical)" port, this module will increase an internal counter say k (k = 0..(N-1)) and transmit a true signal to the corresponding "out-k" output port. So when N=10, the output frequency will be 1/10 of the input frequency.

Properties: "moduleColor" set the background color. "N(2-32)" set the number of the output port. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: TimeObj

Input: None.

Output: "output" date port on the right edge with unlimited connections.

Function: will send out date event every one second. Use the DateObj to show the current time and get the seconds, minutes, etc.

Properties: "moduleColor" set the background color.



Name: TimerObj

Input: "enabled(logical)" and "interval" integer port on the top edge with maximum one connection.

Output: "output" integer port on the right edge and "interval" integer port on the bottom edge with unlimited connections.

Function: When "enabled" is true (default is true), this module will send out integer pulse to "output" port event every "interval" millisecond. The "interval" input and output ports are for duplex communication to external integer objects. So the value will keep consist when user modify the interval by property dialog box.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF). "delay" set the interval value in millisedond.



Name: CalibrateObj

Input: "calibrate(logical)", "lower" and "upper" integer port on the top edge and "input", "feedback" integer port on the left edge with maximum one connection.

Output: "output" and "decalibrate" integer port on the right edge and "lower", "upper" integer port on the bottom edge with unlimited connections.

Function: When "calibrate" is true, this module will adjust to the current input range of "input" and re-map the value into N bits value and broadcast it to "output" port. When "calibrate" is false, this module will not adjust to the current input range, but keep re-map the value by using the current lower and upper value. If the N bits value comes in from the "feedback" port, this module will map it back to the input range (the reverse mapping). The "lower", "upper" input and output ports are for duplex communication to external integer objects. So the value will keep consist when user modify the "lower" and "upper" value by property dialog box.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF). "fraction" set the current blank display area (on the top and bottom of display area) in percent. percent. For example, fraction=0 will use all display area, fraction=25 will have 25% blank area on the top and bottom of display area. "upper" and "lower" set the current input range set.



Name: AvgFilterObj

Input: "input" on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: When event comes into "input" port, this module will calculate the average value of last N input value (include the current value) and broadcast the value through the "output" port. Basically, it works as a low-pass filter.

Properties: "moduleColor" set the background color. "N" set the number of contiguous input values to average.



Name: DelaySustainObj

Input: "input" on the left edge with maximum one connection. "sampling-clock", "delay", and "sustain" on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections. "delay" and "sustain" on the bottom edge with unlimited connections.

Function: When there is a raise (false-true) event detected in "input" port, this module will delay the input value for "delay" ticks (Tick marks come from sampling-clock). When there is drop (true-false) event detected in "input" port, this module will sustain last value for "sustain" ticks. The "delay", "sustain" input and output ports are for duplex communication to external integer objects. So the value will keep consist when user modify the "delay" and "sustain" value by property dialog box.

Properties: "moduleColor" set the background color. "delay" set the number of delay ticks. "sustain" set the number of sustain ticks.



Name: RtoIObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Convert the "input" port's real number into integer (with rounding) and broadcast the result to "output" port.

Properties: "moduleColor" set the background color.



Name: ItoRObj

Input: "input" integer port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Convert the "input" port's integer into real number and broadcast the result to "output" port.

Properties: "moduleColor" set the background color.



Name: RGreaterThanObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is larger "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: RGreaterEqualObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is greater or equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: REqualObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: RNotEqualObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Compare the value of "input-1" and "input-2" and broadcast the result value to "output" port when receiving an input event. Result value is true when "input-1" is not equal "input-2", false otherwise.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it becomes the logical operation (true=1, false=0). When it is 16, it will become the 16 bits operation (true=0xFFFF, false=0x0000).



Name: RAddObj

Input: "input" real number port on the real number edge with unlimited connections.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate the sum of all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RMultiplyObj

Input: "input" real number port on the left edge with unlimited connections.

Output: "output" real number port on the right edge with unlimited connections.

Function: Multiply all the input values and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RSubstractObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate "input-1" - "input-1" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RDivideObj

Input: "input-1" real port on the left edge and "input-2" real port on the top edge with maximum one connection.

Output: "output" real port on the right edge with unlimited connections.

Function: Calculate "input-1" / "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RAbsObj

Input: "input" real number port on the left with maximum one connections.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ABS("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.

 



Name: RPowObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" integer port on the right edge with unlimited connections.

Function: Calculate "input-1" ^ "input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RMaxObj

Input: "input" real number port on the left edge with unlimited connections.

Output: "output" real number port on the right edge with unlimited connections.

Function: Choose the maximum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RMinObj

Input: "input" real number port on the left edge with unlimited connections.

Output: "output" real number port on the right edge with unlimited connections.

Function: Choose the minimum input value and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RSinObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate SIN("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RCosObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate COS("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RASinObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ASIN("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RACosObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ACOS("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RTanObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate TAN("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RATanObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ATAN("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RExpObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate EXP("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RLogObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate LOG("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RSqrtObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate SQRT("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RRoundObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ROUND("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RCeilObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate CEIL("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RFloorObj

Input: "input" real number port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate FLOOR("input") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RModObj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate the reminder of "input-1"/"input-2" and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RATan2Obj

Input: "input-1" real number port on the left edge and "input-2" real number port on the top edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: Calculate ATAN2("input-1", "input-2") and broadcast the result value to "output" port when receiving an input event.

Properties: "moduleColor" set the background color.



Name: RPIObj

Input: None.

Output: "output" real number port on the right edge with unlimited connections.

Function: Return PI value when requested.

Properties: "moduleColor" set the background color.



Name: REObj

Input: None.

Output: "output" real number port on the right edge with unlimited connections.

Function: Return e value when requested.

Properties: "moduleColor" set the background color.



Name: RRandomObj

Input: "control(logical)" integer port on the left edge with maximum one connection.

Output: "output" real number port on the right edge with unlimited connections.

Function: When detecting a raise signal (from false to true) on the "control" port, this module will prepare a real random number (0-1.0) and broadcast the value to "output" port.

Properties: "moduleColor" set the background color.



Name: RCalibrateObj

Input: "calibrate(logical)" integer, "lower" and "upper" real number port on the top edge and "input" real number, "feedback" integer port on the left edge with maximum one connection.

Output: "output" integer and "decalibrate" real number port on the right edge and "lower", "upper" real number port on the bottom edge with unlimited connections.

Function: When "calibrate" is true, this module will adjust to the current input range of "input" and re-map the value into N bits integer value and broadcast it to "output" port. When "calibrate" is false, this module will not adjust to the current input range, but keep re-map the value by using the current lower and upper value. If the N bits value comes in from the "feedback" port, this module will map it back to the input range (the reverse mapping). The "lower", "upper" input and output ports are for duplex communication to external integer objects. So the value will keep consist when user modify the "lower" and "upper" value by property dialog box.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF). "fraction" set the current blank display area (on the top and bottom of display area) in percent. For example, fraction=0 will use all display area, fraction=25 will have 25% blank area on the top and bottom of display area. "upper" and "lower" set the current input range set.



Name: SKeyboardObj

Input: "input" string port on the left edge with maximum one connection.

Output: "output" string number port on the right edge with unlimited connections.

Function: This module will generate string event with exact one character when user press a key. It works even NeatTools is minimized or in the background. If user supply a string event into the "input" port, it will simulate the keyboard event (just like someone is typing the keyboard) to the windows system. This function will only work when NeatTools is in background to avoid feedback problems.

Properties: "moduleColor" set the background color.



Name: KeyboardObj

Input: "stroke(logical)" integer port on the left edge and "depress(logical)" integer port on the top edge with maximum one connection.

Output: "output" string number port on the right edge with unlimited connections.

Function: This module will generate integer event when a key event that matches this module pressed. It works even NeatTools is minimized or in the background. If user supply a true integer event into the "stroke(logical)" port, it will simulate a stroke keyboard event (just like someone press and then release the key) to the windows system. If user supply a raise integer event (from false to true) into the "depress(logical)" port, it will simulate a depress keyboard event (just like someone press the key) to the windows system. If user supply a drop integer event (from true to false) into the "depress(logical)" port, it will simulate a release keyboard event (just like someone release the key) to the windows system. These functions will only work when NeatTools is in background to avoid feedback problems.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MixerObj

Input: "input-L/Mono" integer port on the left edge and "input-R" integer port on the top edge with maximum one connection.

Output: "output-L/Mono" integer port on the right edge and "output-R" integer port on the bottom edge with unlimited connections.

Function: This module is designed to couple the windows' multimedia control components. The "Output Volume" module will accept mono or stereo input value to change the volume level of a sound source. It also will feedback with output value that indicate the volume level of a particular sound source. It will couple the exist mixer applications. So when user change the level value in other mixer application, this module will reflect the change by send out events from output ports. When the module show gray text, it indicate that your system does not support this type of sound source.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MixerObj

Input: "input " integer port on the left edge with maximum one connection.

Output: "output" integer port on the bottom edge with unlimited connections.

Function: This module is designed to couple the windows' multimedia control components. The "Output Mute" module will accept input value to change the state (Mute or not) of a sound source. It also will feedback with output value that indicate the state of a particular sound source. It will couple the existing mixer applications. So when user change the state of a sound source in other mixer application, this module will reflect the change by send out events from output port. When the module show gray text, it indicate that your system does not support this type of sound source.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MixerObj

Input: "input-L/Mono" integer port on the left edge and "input-R" integer port on the top edge with maximum one connection.

Output: "output-L/Mono" integer port on the right edge and "output-R" integer port on the bottom edge with unlimited connections.

Function: This module is designed to couple the windows' multimedia control components. The "Input Volume" module will accept mono or stereo input value to change the recording level of a sound source. It also will feedback with output value that indicate the recording level of a particular sound source. It will couple the existing mixer applications. So when user change the level value in other mixer application, this module will reflect the change by send out events from output ports. When the module show gray text, it indicate that your system does not support this type of sound source for recording.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MixerObj

Input: "input " integer port on the left edge with maximum one connection.

Output: "output" integer port on the bottom edge with unlimited connections.

Function: This module is designed to couple the windows' multimedia control components. The "Input Mute" module will accept input value to change the state (Mute or not) of a sound source's recording. It also will feedback with output value that indicate the state of a particular sound source's recording. It will couple the existing mixer applications. So when user change the state of a sound source in other mixer application, this module will reflect the change by send out events from output port. When the module show gray text, it indicate that your system does not support this type of sound source's mute operation.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MIDIOutObj

Input: "input" MIDI port on the left edge and "enabled(logical)" integer port on the top edge with maximum one connection.

Output: None.

Function: When "enabled" is true, this module will open the assigned MIDI device and start to wait for MIDI event coming from the "input" port. When receive a MIDI event, this module will transfer it into the assigned and opened MIDI device.

Properties: "moduleColor" set the background color. "device" set the destination MIDI device. Depend on windows' setting. It could be internal FM or external MIDI ports. If it is internal FM, send in MIDI events will generate sound. If it is external MIDI port, send in MIDI events could control external MIDI instrument, which connected to this hardware MIDI port.



Name: MIDIInObj

Input: "enabled(logical)" integer port on the top edge with maximum one connection.

Output: "output" MIDI port on the right edge with unlimited connections.

Function: When "enabled" is true, this module will open the assigned MIDI device and start to send out MIDI events (which coming from the hardware MIDI port or the internal MIDI device) to the "output" port.

Properties: "moduleColor" set the background color. "device" set the destination MIDI device. Depend on windows' setting. It could be internal MIDI devise or external MIDI ports. If it is external MIDI port, play notes or playback on external MIDI instrument will send in MIDI events and broadcast through "output" port.



Name: MIDIObj

Input: "midi-in" MIDI port on the top edge with maximum one connection. 127 notes inputs (from "Octave[0] C" to "Octave[10]G"), "channel(0-15)", "channel_pressure", "program(0-127)", "volume", "pan", "damper" and "pitch" integer port on the left edge with maximum one connection.

Output: "midi-out" MIDI port on the bottom edge with unlimited connections. 127 notes outputs (from "Octave[0] C" to "Octave[10]G"), "channel(0-15)", "channel_pressure", "program(0-127)", "volume", "pan", "damper" and "pitch" integer port on the right edge with unlimited connections.

Function: In order to generate MIDI event from integer event, user can use this module to create almost all type of MIDI events. When receive a MIDI event through "midi-in" port, this module will decompose the MIDI event into integer event into the related output ports.

Properties: "moduleColor" set the background color. "N Bits" set the operation width. When it is one, it will generate only logical (1-bit) value (true, or false). When it is 16, it will generate the 16 bits wise value (from 0 to 0xFFFF).



Name: MIDIChannelObj

Input: "midi-in" MIDI port on the top edge with maximum one connection. "ch1-in", "ch2-in" ... , "ch15-in", and "misc-in" MIDI port on the left edge with maximum one connection.

Output: "midi-out" MIDI port on the bottom edge with unlimited connections. "ch1-out", "ch2-out" ... , "ch15-out", and "misc-out" MIDI port on the right edge with unlimited connections.

Function: There are 16 channel in the MIDI signal. When receive MIDI event from the "midi-in" port, this module can separate the event into the correspond output channel port. If user want a MIDI event change to anther channel, he can feed the MIDI event to say the "ch1-in" port. Then the even coming out from the "midi-out" port will have MIDI signal with channel 1 assigned.

Properties: "moduleColor" set the background color.



Name: MIDIFileObj

Input: "record(logical)" integer, "play(logical)" integer, "pause(logical)" integer, "filename" string, "sequence" integer, "pos" integer, "N(1-128)" integer, and "ratio" integer port on the top edge with maximum one connection. There are MIDI ports (name and number of the MIDI ports determined by MIDI file user assigned) on the left edge with maximum one connection.

Output: "count", "total", and "tempo" integer port on the bottom edge with unlimited connections. There are MIDI ports (name and number of the MIDI ports determined by MIDI file user assigned) on the right edge with unlimited connections.

Function: This module could load a MIDI file with "MID" extension and play it back or record(Recording portion is not complete yet). When "record(logical)" receive a raised event (from false to true), it will start to recording MIDI events into assigned file. When "play(logical)" receive a raised event, it will start to play back the content of the current MIDI file. When "pause" is true, it will pause the current recording or playback action. The "filename" and "sequence" combine could use to assign the MIDI filename. When "sequence" is zero, the "filename" is the assigned filename. If "sequence" not equal to zero, say 1, the assigned filename will become "filename_1". Just change the sequence number and user could record on sequences of different files. The "pos" is the start event count. So user could playback from any starting position. The "N" set the

Properties: "moduleColor" set the background color.

2. NeatTools Class Hierarchy:

+-JObject

|

+-JClipboard

|

+-JColor

|

+-JDimension

|

+-JEvent

|

+-JFontMetrics

|

+-JInsets

|

+-JLayoutManager

| |

| +-JBorderLayout

| |

| +-JFlowLayout

| |

| +-JGridLayout

|

+-JPoint

| |

| +-JRect

| |

| +-JComponent

| |

| +-JCanvas

| | |

| | +-JLabel

| | | |

| | | +-JButton

| | | | |

| | | | +-JPushButton

| | | | | |

| | | | | +-JThumb

| | | | | |

| | | | | +-JTriangleButton

| | | | |

| | | | +-JToggleButton

| | | | |

| | | | +-JModuleButton

| | | |

| | | +-JTextField

| | |

| | +-JListBox

| | |

| | +-JRuler

| | |

| | +-JScroller

| | |

| | +-JSeparator

| | |

| | +-JAbout

| | |

| | +-JModuleCanvas

| |

| +-JDialog

| |

| +-JModal

| | |

| | +-JMessageBox

| | |

| | +-JColorBox

| | |

| | +-JInputBox

| | | |

| | | +-JFileBox

| | | |

| | | +-JIntegerBox

| | |

| | +-JIntegerListBox

| | |

| | +-JPropertyBox

| |

| +-JPanel

| |

| +-JWindow

|

+-JBlockInputStream

|

+-JBlockOutputStream

|

+-JFile

|

+-JFilterInputStream

| |

| +-JBufferedInputStream

|

+-JFilterOutputStream

| |

| +-JBufferedOutputStream

|

+-JPipedStream

|

+-JBoolean

|

+-JCharacter

|

+-JDouble

|

+-JFloat

|

+-JInteger

|

+-JLong

|

+-JMath

|

+-JPObject

| |

| +-JObjectPtr

|

+-JReference

| |

| +-JFont

| |

| +-JGraphics

| |

| +-JImage

| |

| +-JRegion

| |

| +-JBlock

| |

| +-JCriticalSection

| |

| +-JDescriptor

| | |

| | +-JColor

| | |

| | +-JFileInputStream

| | | |

| | | +-JFileIOStream

| | |

| | +-JFileOutputStream

| | |

| | +-JSocket

| | | |

| | | +-JServerSocket

| | |

| | +-JSocketInputStream

| | |

| | +-JSocketOutputStream

| |

| +-JProcess

| |

| +-JString

| |

| +-JThread

| |

| +-JArray

| |

| +-JList

| |

| +-JDList

|

+-JSystem

|

+-JThrowable

| |

| +-JError

| |

| +-JException

| |

| +-JIOException

| | |

| | +-JEOFException

| | |

| | +-JInterruptedIOException

| | |

| | +-JClassReferenceException

| | |

| | +-JSocketException

| | |

| | +-JUnknownHostException

| |

| +-JRuntimeException

| |

| +-JArithmeticException

| |

| +-JIllegalArgumentException

| |

| +-JNullPointerException

| |

| +-JProcessCreateException

| |

| +-JThreadCreateException

|

+-JDataType

| |

| +-JBlockData

| | |

| | +-JBytesData

| | |

| | +-JVideoData

| | |

| | +-JWaveData

| |

| +-JIntegerData

| | |

| | +-JColorData

| | |

| | +-JDateData

| | |

| | +-JMIDIData

| |

| +-JRealData

| |

| +-JStringData

|

+-JFDimension

|

+-JFPoint

| |

| +-JFRect

| |

| +-JViewObj

| |

| +-JGuideObj

| |

| +-JLineObj

| |

| +-JModuleObj

| | |

| | +-JColorObj

| | |

| | +-JLEDObj

| | | |

| | | +-JLabelObj

| | | | |

| | | | +-JDateObj

| | | | |

| | | | +-JIntegerObj

| | | | |

| | | | +-JRealObj

| | | |

| | | +-JNBitsObj

| | | |

| | | +-J1DMeterObj

| | | | |

| | | | +-J1DViewerObj

| | | |

| | | +-J2DMeterObj

| | | |

| | | +-JBtnObj

| | | |

| | | +-J1DSliderObj

| | | |

| | | +-J2DSliderObj

| | | |

| | | +-JFocusObj

| | | |

| | | +-JPushBtnObj

| | | |

| | | +-JSwitchObj

| | |

| | +-JRAddObj

| | | |

| | | +-JRAbsObj

| | | | |

| | | | +-JItoRObj

| | | | |

| | | | +-JRACosObj

| | | | |

| | | | +-JRASinObj

| | | | |

| | | | +-JRATanObj

| | | | |

| | | | +-JRCeilObj

| | | | |

| | | | +-JRCosObj

| | | | |

| | | | +-JRDivideObj

| | | | | |

| | | | | +-JRAtan2Obj

| | | | | |

| | | | | +-JRModObj

| | | | | |

| | | | | +-JRPowObj

| | | | | |

| | | | | +-JRSubtractObj

| | | | |

| | | | +-JRExpObj

| | | | |

| | | | +-JRFloorObj

| | | | |

| | | | +-JRLogObj

| | | | |

| | | | +-JRRandomObj

| | | | |

| | | | +-JRRoundObj

| | | | |

| | | | +-JRSinObj

| | | | |

| | | | +-JRSqrtObj

| | | | |

| | | | +-JRTanObj

| | | | |

| | | | +-JRtoIObj

| | | |

| | | +-JRMaxObj

| | | |

| | | +-JRMinObj

| | | |

| | | +-JRMultiplyObj

| | | |

| | | +-JRPIObj

| | | |

| | | +-JREObj

| | |

| | +-JTimeObj

| | |

| | +-JAddObj

| | |

| | +-JAbsObj

| | | |

| | | +-JDelayObj

| | | |

| | | +-JDivideObj

| | | | |

| | | | +-JAccumulatorObj

| | | | |

| | | | +-JPowObj

| | | | |

| | | | +-JRemainObj

| | | | |

| | | | +-JSubtractObj

| | | |

| | | +-JNodeObj

| | |

| | +-JAvgFilterObj

| | |

| | +-JBtoIObj

| | |

| | +-JCOMObj

| | |

| | +-JComplexObj

| | |

| | +-JConvertObj

| | |

| | +-JDataBaseObj

| | |

| | +-JDaviconObj

| | |

| | +-JDeMultiplexerObj

| | |

| | +-JDecoderObj

| | |

| | +-JDelaySustainObj

| | |

| | +-JEncoderObj

| | |

| | +-JExclusiveObj

| | |

| | +-JItoBObj

| | |

| | +-JLPTObj

| | |

| | +-JMIDIChannelObj

| | |

| | +-JMIDIOutObj

| | | |

| | | +-JMIDIInObj

| | |

| | +-JMaxObj

| | |

| | +-JMinObj

| | |

| | +-JMultiplexerObj

| | |

| | +-JMultiplyObj

| | |

| | +-JRecorderObj

| | | |

| | | +-JMIDIFileObj

| | | | |

| | | | +-JRMIDFileObj

| | | |

| | | +-JWaveFileObj

| | |

| | +-JSampleObj

| | | |

| | | +-JControlObj

| | |

| | +-JSocketObj

| | | |

| | | +-JServerSocketObj

| | |

| | +-JWaveOutObj

| | | |

| | | +-JWaveInObj

| | |

| | +-JANDObj

| | |

| | +-JCalibrateObj

| | |

| | +-JClockDividerObj

| | |

| | +-JJoyStickObj

| | |

| | +-JMIDIObj

| | |

| | +-JMixerObj

| | |

| | +-JNOTObj

| | | |

| | | +-JCHObj

| | | |

| | | +-JDGreaterThanObj

| | | | |

| | | | +-JDEqualObj

| | | | |

| | | | +-JDGreaterEqualObj

| | | | |

| | | | +-JDNotEqualObj

| | | |

| | | +-JGreaterThanObj

| | | | |

| | | | +-JEqualObj

| | | | |

| | | | +-JGreaterEqualObj

| | | | |

| | | | +-JNotEqualObj

| | | |

| | | +-JKeyboardObj

| | | |

| | | +-JMouseBtnObj

| | | | |

| | | | +-JMousePosObj

| | | |

| | | +-JMouseObj

| | | |

| | | +-JPulseObj

| | | |

| | | +-JRGreaterThanObj

| | | | |

| | | | +-JREqualObj

| | | | |

| | | | +-JRGreaterEqualObj

| | | | |

| | | | +-JRNotEqualObj

| | | |

| | | +-JRandomObj

| | | |

| | | +-JSGreaterThanObj

| | | |

| | | +-JSEqualObj

| | | |

| | | +-JSGreaterEqualObj

| | | |

| | | +-JSNotEqualObj

| | |

| | +-JORObj

| | |

| | +-JOxfordObj

| | |

| | +-JRCalibrateObj

| | |

| | +-JTNGObj

| | | |

| | | +-JTNG3Obj

| | |

| | +-JTimerObj

| | |

| | +-JXORObj

| |

| +-JViewSet

| |

| +-JFocusSet

| |

| +-JLinkObj

|

+-JProperty

| |

| +-JColorProperty

| |

| +-JIntegerListProperty

| |

| +-JIntegerProperty

| |

| +-JRealProperty

| |

| +-JStringProperty

| |

| +-JFileProperty

|

+-JInetAddress

|

+-JAssociation

|

+-JDataBase

|

+-JDate

|

+-JFileArray

|

+-JHashTable

| |

| +-JDictionary

|

+-Jrandom

 

3. Java-like API package
1. LANG package

The LANG package contains the classes that make up the core of the Java-like API layer. The classes of the LANG package are summarized in the following table:

Class	Description
JBlock	Memory block class
JBoolean	Object wrapper for Boolean values
JCharacter	Object wrapper for char values
JCriticalSection	Exclusive object for critical section
JDescriptor	Generic descriptor manipulate object
JDouble	Object wrapper for double values
JFloat	Object wrapper for float values
JInteger	Object wrapper for integer values
JLong	Object wrapper for long values
JMath	Object wrapper for math functions
JObject	Generic object class, at top of inheritance hierarchy
JObjectPtr	Object wrapper for object pointer
JPObject	Object wrapper for object reference
JProcess	Generic behavior for process
JReference	Generic handle manipulate object
JString	Character strings
JThread	Class for managing threads
JThrowable	Generic exception class; all object thrown must be a Throwable

Table 2: Classes of LANG package

2. UTIL package

The UTIL package contains various utility classes, including random numbers, system properties, and other useful classes. Classes of the UTIL package are summarized in the following table:

Class	Description
JArray	A array of objects
JAssociation	Association object that associate two objects
JDList	A double link list
JDictionary	A class that maps between keys and values
JFileArray	A massive array that store content in file system
JHashTable	A hash table
JList	A link list
JRandom	A random number generator object

Table 3: Classes of UTIL package

3. IO package

The IO package provides input and output classes and classes for streams and files. The classes of IO packages are summarized in the following table:

Class	Description
JBlockInputStream	An input stream from a memory block
JBlockOutputStream	An output stream from a memory block
JBufferedInputStream	A buffered input stream
JBufferedOutputStream	A buffered output stream
JFile	Represents a file on the file system
JFileIOStream	A input/output stream from and to a file
JFileInputStream	An input stream from a file
JFileOutputStream	An output stream to a file
JFilterInputStream	A class which provides a filter for input stream
JFilterOutputStream	A class which provides a filter for output stream
JInputStream	An abstract class representing an input stream
JOutputStream	An abstract class representing an output stream
JPipedStream	An piped input/output stream

Table 4: Classes of IO package

4. NET package

The NET package contains classes for performing network operations, such as sockets and URLs. Classes of the NET package are summarized in the following table:

Class	Description
JInetAddress	An object representation of an Internet host
JServerSocket	A server-side socket
JSocket	A socket
JSocketInputStream	An input stream from a socket connection
JSocketOutputStream	An output stream to a socket connection

Table 5: Classes of NET package

5. AWT package

The AWT package contains the classes that make up the Abstract Windowing Toolkit. This package contains the window abstraction and graphics abstraction related classes. The classes of AWT package are summarized in the following table:

Class	Description
JBorderLayout	A layout manager for arranging items in border formation
JButton	A UI button
JCanvas	A canvas for drawing and performing other graphics operation
JClipboard	An object wrapper for clipboard buffer
JColor	A representation of a color
JColorBox	A dialog box that could change color setting
JComponent	The generic class for all UI components
JDialog	A window for brief interactions with users
JDimension	width and height
JEvent	An object representing events caused by the system or based on user input
JFileBox	A dialog box that could select a file in file system
JFlowLayout	A layout manager that lays out objects from left to right in rows
JFont	A representation of a font
JFontMetrics	A class for holding information about a specific font's character information
JGraphics	The generic behavior for representing a graphics context, and for drawing and painting shapes and objects
JGridLayout	A layout manager with rows and columns; elements are added to each cell in the grid
JImage	A representation of a bitmap image
JInputBox	A dialog box for input text
JInsets	Distances from the outer border of the window; used to layout components
JIntegerBox	A dialog box for input integer values
JIntegerListBox	A dialog box for select integer values
JLabel	A text label for UI components
JLayoutManager	A generic class for all layout manager
JListBox	A scrolling list
JMessageBox	A dialog box that display a message to users
JModal	A specialized dialog box that will interact with users
JPanel	A container that is displayed
JPoint	x and y coordinates
JPushButton	A UI pushbutton
JRect	x and y coordinates for the top corner, plus width and height
JRegion	A logical region defined in graphics context
JRuler	A UI ruler
JScroller	A UI scrollbar
JSeparator	A UI separator
JTextField	A fixed-size editable text field
JThumb	A UI thumb in scrollbar
JToggleButton	A UI toggle button
JTriangleButton	A UI triangle button in scrollbar
JWindow	A top-level window

Table 6: Classes of AWT package

4. NeatTools application package

The NeatTools application package is further decomposed into NEAT, DESKTOP, Modules, and External Modules packages as show in Figure 5:



Figure 5: NeatTools application layer

1. NEAT package



Figure 5: Property in NEAT package



Figure 7: NEAT package (continue)

The NeatTools modules could define internal properties, which could be modified through a property dialog box. Each property will become an item in a property list box. A user could select and modify them. Taking advantage of the object oriented programming technique, the Property class becomes the generic property representation. And all other properties will be derived from Property class and will benefit the unified process and editing procedures.

Just as the LANG package is the core of Java-like API layer, the NEAT package is the core of the NeatTools application layer. It defines the basic object structure of how JViewObj aggregates into JVewSet. It also defines the basic module behavior which includes input group and output group (an aggregation of JFocusSet. Here, JFocusSet is the composition of JLinkObj that represents the input and output links of a module object). Because of the polymorph feature of object orient programming, the JViewSet could become the composition of any object that derived from JVewObj. This structure forms a container-component relationship pattern.

The classes of NEAT package are summarized in the following table:

Class	Description
JANDObj	A NeatTools module that handle AND operation
JAbout	A banner that shows the NeatTools
JColorProperty	A property that represent color values
JFDimension	width and height in floating point format
JFPoint	x and y in floating point format
JFRect	A rectangle in floating point format
JFileProperty	A property that represent files
JFocusSet	A composition of JViewObj reference
JGuideObj	A guide line in NeatTools desktop area
JIntegerListProperty	A property that represent a list of integer values
JIntegerProperty	A property that represent integer values
JLineObj	A line representation in NeatTools desktop area
JLinkObj	A link between two NeatTools modules
JModuleButton	A UI button that display NeatTools modules
JModuleCanvas	A UI canvas that display NeatTools modules
JModuleObj	A generic class of all NeatTools modules
JProperty	A generic class of all properties
JPropertyBox	A dialog box that could change the property settings
JRealProperty	A property that represent the floating point values
JStringProperty	A property that represent the string values
JViewObj	A generic class of all display object in NeatTools desktop area
JViewSet	A composition of JViewObj objects

Table 7: Classes of NEAT package

1. Modules package

All NeatTools modules are derived directly or indirectly from JModuleObj. Currently, there are around 100 different modules in NeatTools. Each module works like a black box, or ICs, which could handle one or several functions in a particular field. As AND module is the AND gate in digital logic, OR module is the OR gate in digital logic. Please see section 3 for more detailed information on NeatTools module structure.

2. External modules package

The external modules are just like a normal NeatTools module. The only difference is how it is complied and linked into the system. Follow the special instructions in "NeatTools external module development kit" and it will guide you through the creation of an external module and into a dynamic linkage library (DLL). NeatTools will load the external modules in startup when it can locate them in the NeatTools directory. Or they could be loaded in runtime by pressing "Load Module" button in NeatTools. If an external module is used in a NeatTools network and the module was not loaded in memory yet, NeatTools will popup a dialog box for the user to specify the location of the particular external module.

3. DESKTOP package

The DESKTOP package contains a very complex class called JView, which is a UI canvas as well as the NeatTools desktop area. It contains the aggregation and composition of all the NeatTools modules, links between modules, guide lines, and focus objects selected by users. It also handles all users and system events, such as mouse events, keyboard events, and system notification events. It represents and implements module display, module layout, grid snapping, thread operation exclusive, module persistency, module creation, data flow network construction, clipboard buffer operations, and event broadcast initialization.

2. Module Programming Introduction
1. Module Event Broadcast Model

By using module abstraction, I simplify the module event broadcast model into a process involve only three methods and directly function call between these methods. These three methods include engine, broadcast, and access. Usually, programmer will have to override engine and access method to define the module's behavior. The behavior here means how modules react to external events and change its internal state to respond to the event. Also, some module could change its state or broadcast message without external trigger by using thread.

How those three method methods work together? And, how NeatTools system involved in the process? First, I have to mansion the Link object, which NeatTools use to record the logical connection between modules. The object defined as follow (Programmer always could find all the header files under http://www.pulsar.org/ej/work/oop/oop.html:

class

JLinkObj : public JViewSet {

...

public:

...

JLinkObj();

...

void access(const JDataType& data);

void engine();

...

JModuleObj *from, *to;

int fn, tn;

};

 

 

In this object, I defined the public data member include the Module pointer 'from' and 'to', and Interger 'fn' and 'tn'. Pointer 'from' reference the source module and Pointer 'to' reference the destination module. 'fn' stand for the output port number of the source module and 'tn' stand for the input port number of the destination module. In NeatTools module, programmer can define the number of input or output by fixed number or dynamically change the number of input or output port. Each port could accept one or more connections depend on the behavior of module. I will discuss that later on. So what happen when user make connection between two modules? NeatTools will initiate a link instance and put it in a link list of links and then each module on the particular port will have separate link list, which has the reference to the link itself. (For performance reason, so module could direct access to the link object without search).

Now, Lets get back to the broadcast model again. Module programmer could decide to broadcast event on a particular port at any time. It could happen in its thread loop or just respond to incoming event from other modules. When broadcast(fn) is called, NeatTools will invoke all the connected module's engine(tn, link). When a module's engine method got invoked, it knows an event occur and it comes from which module, which output port, and it goes into which input port by access the data member of the link parameter. When programmer implement the engine method, he will first use tn to identify which input port got event and then use link.access(data) to access the event content. the link.access(data) will translate into call source module's access(fn, link, data) method. So before call broadcast, programmer should prepare the value in data member so when the destination module call its access method, it could provide the value right on time. Why, we have this trouble to have the engine call the access in order to get the data? Because, in some situation, like AND gate, its input port could accept any number of connections. And when an event occur through one of the connection, the AND has to go through all the input connection to figure out the result value. In this case, The AND gate call module's access even when that module did not call broadcast. So, programmer should keep it in mind that access should always provide a current value.

In order to see how it works, let see a simple example on a module which calculate X² as an output when integer event going in. We will just focus on the broadcast related methods at this time.

// JSquareObj.h

class JSquareObj : public JModuleObj {

protected:

virtual void writeContent(class JOutputStream& os);

virtual void readContent(class JDictionary& dict);

public:

virtual const char* className() const;

virtual JObject* clone() const;

JSquareObj();

virtual void draw(JGraphics g, int x, int y, int w, int h);

virtual void access(int n, JLinkObj& link, const JDataType& data);

virtual boolean inputAllowed(int n);

virtual boolean outputAllowed(int n);

virtual void engine(int n, JLinkObj& link);

protected:

int value;

};

 

#include "JSquareObj.h"

#include "JLinkObj.h"

#include "JIntegerData.h"

 

char* theJSquareObj = JSquareObj().Register();

void JSqureObj::writeContent(JOutputStream& os) {

JModuleObj::writeContent(os);

putInteger(os, "value", value);

}

 

void JSquareObj::readContent(JDictionary& dict) {

JModuleObj::readContent(dict);

value = getInteger(dict, "value");

}

 

const char* JSquareObj::className() const { return "JSquareObj";}

JObject* JSquareObj::clone() const { return new JSquareObj(*this);}

JSquareObj::JSquareObj() { igm = 1; value = 0;}

 

void JSquareObj::draw(JGraphics g, int x, int y, int w, int h) {

JRect rect(x, y, w, h);

drawText(g, JString("Sqr"), rect);

}

 

void JSquareObj::access(int n, JLinkObj& link, const JDataType& data) { INT(data) = value;}

boolean JSquareObj::inputAllowed(int n)

{ return !inputSet(n).last();}

boolean JSquareObj::outputAllowed(int n) { return true;}

 

void JSquareObj::engine(int n, JLinkObj& link) {

int nv;

link.access(JIntegerData(nv));

nv *= nv;

if (value != nv) {

value = nv;

broadcast(0);

}

}

 

Here, we start our focus on engine method. In this method, we declare a temporary integer variable nv. the link.access(JIntegerData(nv)) will first create JIntegerData (a derived class of JDataType) instance and use it as a media to transfer integer value into nv. Every data type is derived class of JDataType. I will discuss the data type polymorphism later. Because of polymorphism, we can use the same access method to transfer all kind of different data type. After this statement, the input value will assign on variable nv. So we calculate the square of nv. If nv and value are different, we update the value and call broadcast(0). Here we only have one output port, so the only valid broadcast number would be zero. When process broadcast method, the destination module's engine method will get invoked and eventually it will call the access method of source module. Here, in access method, we have only one statement: INT(data) = value. INT is a macro defined in JIntegerData.h to type cast JIntegerData into int&. So we can use it and assign new value. Programmer also could use data.assign(JInteger(value)). JInteger is a wrapper class for int data (Like the Integer class in Java). But the first implementation is faster.

2. Basic Methods in Module

Almost every object class in NeatTools is a subclass of JObject directly or indirectly. There are some method which always need to be take care of or override to ensure the basic object behavior works correctly.

In the previous example module, after invoke some header files, you will see a strange line :

char* theJSquareObj = JSquareObj().Register();

This statement will call the Register method and register the module into NeatTools system. So later on, NeatTools could duplicate or recreate object instance by using this registered copy. If you fail to define this line, your module will become invisible to NeatTools system.

Programmer also need to override the className() and clone() methods. In NeatTools, I use the character string return by className() method to identify different objects. Please make sure not to define different class with the same name return by className(). The clone method will new and return an object instance. If you fail to define these methods, NeatTools could not generate the module you designed.

The constructor is an option. But usually you will initialize your data member or set the default input or output port number. 'igm' and 'ogm' are the variable represent the current input and output port number. In this example, I set 'igm' to 1 and the default value of 'ogm' (defined in JModuleObj) is 1 also. So here, this module has only one input and output port. If you try to change the number of input or output port respond to property or input event, there are some complex issues on layout refresh problem. Please refer to the source code in NeatTools Module develop kit and find out how to implement this feature.

3. Information Methods in Module

NeatTools need more information on input and output port. For example the data type, the attached edge, the tag, etc. The following methods will need to be override: (If you have some module, which has common behavior, you can use one of the class as base class and use it to derive other class. This way, we could save time and space.)

1. JString inputTag(int n), JString outputTag(int n)

NeatTools use these methods to display the tag or description for a particular input or output port. The default behavior in Module is following:

JString JModuleObj::inputTag(int n) {

char* tag[] = { "input", "enable(logical)"};

return tag[n];

}

 

JString JModuleObj::outputTag(int n)

{ return "output";}

 

So when user move to the port area, NeatTools system will display the tag by using the return value of this method.

2. int inputType(int n), int outputType(int n)

Each input or output port could associate with different data type. Here we can return different data types that identify different ports. Or just return one data type that identify all ports. The default behavior is following:

int JModuleObj::inputType(int n) { return JIntegerData::id;}

int JModuleObj::outputType(int n) { return JIntegerData::id;}

 

Here, it defines input and output ports are of integer type. If you want to use other data type, please see the header files in NEAT directory. If you want to set different data type for each different port, you can write something like this:

int JTestObj::inputType(int n) {

switch (n) {

case 0: return JIntegerData::id;

case 1: return JStringData::id;

case 2: return JWaveData::id;

}

return JIntegerData::id;

}

 

3. boolean inputAllowed(int n), boolean outputAllowed(int n)

These two methods define the accessibility of an input or an output port. When it returns true, NeatTools could add new connection to it. When it returns false, NeatTools would indicate the port is already occupied. The default behavior is following:

boolean JModuleObj::inputAllowed(int n) { return false;}

boolean JModuleObj::outputAllowed(int n) { return false;}

 

So if you derive your module directly from JModuleObj, you will have to override these methods to ensure your input or output port could accept new connections. If you want input port accept only one connection, you can do something like this:

boolean JSquareObj::inputAllowed(int n)

{ return !inputSet(n).last();}

 

The inputSet(n) will return the link list of the input port number n. So if link is empty, its last() method will return null. So inputAllowed() will return true and NeatTools could accept new connection for this port. Otherwise, NeatTools will not accept new connection. Programmer could apply the outputSet(n) to the outputAllowed() method.

4. int inputFace(int n), int outputFace(int n)

These two method decide the attach face that a connection would connect to. For inputFace(n), you could return either LEFT or TOP. For outputFace(n), you could return either RIGHT or BOTTOM. The default behavior is following:

int JModuleObj::inputFace(int n) { return (!n)? LEFT : TOP;}

int JModuleObj::outputFace(int n) { return RIGHT;}

 

It means inputFace(0) will return LEFT and inputFace(1) will return TOP. and outputFace() always return RIGHT. So connection to the first input port will attach on left edge, but the second input port will attach on top edge.

5. JFRect inputArea(int n), JFRect outputArea(int n)

These two methods decide the area that a port will accept connections. When user's mouse move within the area defined by JFRect (A floating point rectangle with x, y, with, height data member), NeatTools will show the inverse rectangle to indicate the port area. Module itself is derived from JFRect also. So itself has the x, y, width, and height data member which define the current position and dimension of the module. By using module's with and height value you can define inputArea or outputArea like this:

JFRect JModuleObj::inputArea(int n) {

if (!n) return JFRect(0, height/4, width/6, height/2);

return JFRect(width/4, 0, width/2, height/6);

}

 

JFRect JModuleObj::outputArea(int n)

{ return JFRect(width*5/6, height/4, width/6, height/2);}

 

But, I found out, it is too much trouble for most module with several input port and some of the module need to change the number of input or output port dynamically. So I wrote some utility method which will calculate the rectangle automatically. The default behavior is following:

JFRect JModuleObj::inputArea(int n) {

if (!n) return leftArea(n, 0, 1);

return topArea(n, 1, 1);

}

 

JFRect JModuleObj::outputArea(int n)

{ return rightArea(n, 0, 1);}

 

The leftArea, topArea, rightArea, and bottomArea method are defined in JMoudleObj.

JFRect topArea(int n, int start, int N,

double from = 0.25, double delta = 0.5, int factor = 6);

JFRect bottomArea(int n, int start, int N,

double from = 0.25, double delta = 0.5, int factor = 6);

JFRect leftArea(int n, int start, int N,

double from = 0.25, double delta = 0.5, int factor = 6);

JFRect rightArea(int n, int start, int N,

double from = 0.25, double delta = 0.5, int factor = 6);

 

The 'n' stand for the port number. 'start' means start from which port number. 'N' means for how many port. 'from' stand for the relative position that area start. 'delta' stand for the relative distance that 'N' port will occupy. 'factor' stand for the divide factor of area. If 'factor' is 6, it means area will have height/6 or width/6 dimension. So leftArea(n, 0, 3) means start from port 0 to port 2 (total 3 ports) will occupy area JFRect(0, height/4, width/6, height/2) and divide it into 3 portions from top to down are port0, port1, and port2.

6. JFPoint inputPoint(int n, class JLinkObj& link), JFPoint outputPoint(int n, class JLinkObj& link)

 

These two method is pretty much like the inputArea and outputArea. But the only different is it will return the final attachment point of the connection to a particular port. You will see the link become one of the parameter. It is because, for those port could accept more than one connection, each connection is going to have different attachment point event they are belong to the same port. But using this parameter, we could decide its order and calculate the attachment point when necessary. Again, I wrote the following utility methods. The usage is exactly the same to area utility methods. Only you have to put in the extra link parameter.

JFPoint topPoint(int n, class JLinkObj& link, int start, int N,

double from = 0.25, double delta = 0.5);

JFPoint bottomPoint(int n, class JLinkObj& link, int start, int N, double from = 0.25, double delta = 0.5);

JFPoint leftPoint(int n, class JLinkObj& link, int start, int N, double from = 0.25, double delta = 0.5);

JFPoint rightPoint(int n, class JLinkObj& link, int start, int N, double from = 0.25, double delta = 0.5);

 

4. Display Related Methods in Module

Display is a very important issue in NeatTools. I have been changing the display mechanism several time to increase the throughput especially for high frequency events that involve repaint modules. But there are only a few methods that related to display. Let's discuss paint, update, and repaint method:

void paint(JGraphics g, double dx, double dy, JRegion& rgn, double scale);

void update(JGraphics g, double dx, double dy, JRegion& rgn, double scale);

void repaint();

 

NeatTools will call paint method where there is a need to refresh the graphics contents of module. For example, when user do resize, scroll, scale, etc. operations on the NeatTools windows could cause a module's paint method get invoked. If a module decide to refresh itself and respond to its internal state's change (like LED receive a true event and decide to turn itself on), it could call the repaint() method. Eventually, NeatTools will invoke module's update method. (Depend on the situation, cause module could covered by other module or out of desktop's scope.) Usually, when receive paint, module will draw everything include frame and contents. But when received update, module could only draw the contents. To save space, usually, in paint method we will only draw frame and call the update to draw the contents. But basically, programmer could do whatever he wants depend on the situations. The following is an example on LED object:

void JLEDObj::paint(JGraphics g, double dx, double dy, JRegion& rgn, double scale) {

JRect rect = getIExtent(dx, dy, scale);

g.setJColor(moduleColor);

if ((rect.width > depth2) && (rect.height > depth2))

g.draw3DJRect(rect, depth);

update(g, dx, dy, rgn, scale);

}

 

void JLEDObj::update(JGraphics g, double dx, double dy, JRegion& rgn, double scale) {

JRect rect = getIExtent(dx, dy, scale);

if (value == 0) g.setJColor(bkgnd);

else g.setJColor(color);

if ((rect.width > depth2) && (rect.height > depth2)) {

g.fillJRect(rect.shrink(depth, depth));

} else g.fillJRect(rect);

}

 

The JGraphics object is the graphics context that module could make their graphics operations or setup parameters. It works like a media between program and actual devices. In this case, the JGraphics is always an area on screen, if user make some operation by calling JGraphics' methods, it will reflect the change to screen.

Here, in paint method, first we use getIExtent to get the current module extent in integer screen unit, because NeatTools could display module in any scale. Module will not know its current size on screen until its paint or update methods get invoked. After this statement, the rect contains the current module size in pixel unit. Then, depend on the value itself, if value is false, we set the current color to background color, otherwise we set it to foreground color. If current size is large enough to draw the frame, we call g.draw3DJRect to draw the 3d frame. Then we call update to draw the contents.

In update, we follow the same process and later on fill the LED module with the proper color by using g.fillJRect methods.

void JLEDObj::engine(int n, JLinkObj& link) {

if (!n) {

int iv;

link.access(JIntegerData(iv));

setValue(iv);

} else processColor(n, link);

}

 

boolean JLEDObj::setValue(int _value) {

if (_value != value) {

value = _value;

broadcast(0);

repaint();

return true;

}

return false;

}

 

So, when will LED decide to update itself and make visual feedback to user? Let's look at the engine method. If n is equal to zero (the input port on the left edge, in this case), it will use link.access to assign the integer value into iv variable. And call setValue(iv). In setValue method, we will check if the value need change, if it does, we update the value first and call broadcast(0) and repaint() to refresh its display. Later on, NeatTools will invoke its update method to fill the color when necessary.

The default behavior of display methods in JViewObj (JViewObj is the base class of all objects that could display itself on screen) are:

void JViewObj::paint(JGraphics g, double dx, double dy, JRegion& rgn, double scale) {}

void JViewObj::update(JGraphics g, double dx, double dy, JRegion& rgn, double scale)

{ paint(g, dx, dy, rgn, scale);}

 

Here the paint draw nothing and update call paint directly. So derived class could just override paint for their simple display need.

The default behaviors of display methods in JModuleObj are:

void JModuleObj::draw(JGraphics g, int x, int y, int w, int h) {

JRect rect(x, y, w, h);

drawText(g, JString("Mod"), rect);

}

 

void JModuleObj::paint(JGraphics g, double dx, double dy, JRegion& rgn, double scale) {

g.setJColor(moduleColor);

JRect rect = getIExtent(dx, dy, scale);

g.fill3DRect(rect.x, rect.y, rect.width, rect.height, 2);

g.setJColor(moduleColor.darker());

draw(g, rect.x+3, rect.y+3, rect.width-8, rect.height-8);

g.setJColor(moduleColor.brighter());

draw(g, rect.x+5, rect.y+5, rect.width-8, rect.height-8);

g.setJColor(JColor::black);

draw(g, rect.x+4, rect.y+4, rect.width-8, rect.height-8);

}

 

Here, I define a new method for user who only needs simple display on their module. When system need module's display, it will call paint. In paint, we use color and offset to create the 3D look and feel. So, if user only override draw method and draw some simple text or shapes, it will have the 3D look without repeat the process three times.

5. Module persistency related Methods

One important thing that NeatTools system does is provide user the capacity to layout modules and make connections between modules. But the other task that NeatTools does is equally important - module persistency. That means, NeatTools provide a way to let module store its current state and connections into secondary storage space like hard disk. And later on, retrieve the information and reconstruct every module, restore their state and rebuild their connections. So, how NeatTools achieve the module persistency? NeatTools rely on two methods in module to accomplish this requirement.

protected:

virtual void writeContent(class JOutputStream& os);

virtual void readContent(class JDictionary& dict);

 

The implementation in LED looks like this:

void JLEDObj::writeContent(JOutputStream& os) {

JModuleObj::writeContent(os);

putObject(os, "color", color);

putObject(os, "bkgnd", bkgnd);

putInteger(os, "value", value);

}

 

void JLEDObj::readContent(JDictionary& dict) {

JModuleObj::readContent(dict);

JObject *obj = getObject(dict, "color");

if (obj) color = *(JColor*)obj;

obj = getObject(dict, "bkgnd");

if (obj) bkgnd = *(JColor*)obj;

value = getInteger(dict, "value");

}

 

'writeContent' method will be invoked when NeatTools want a module to write its current states into output stream. Programmer has to call base class's writeConnect method first. Cause every module will only handle the data member that declare in its object scope. Sometimes a data member is just for temporary usage, we could decide not to write their current state. readContent is the opposite to writeContent. When we try to 'put' a variable, we assign a string name to identify the variable. Later on, the readContent provide user a dictionary object, which contain all the available variables. Programmer could use string name as a key to retrive the variables. Please notice that the string name could not contain space or other special characters, use the C++ variable identifier naming syntax as a guideline to define the NeatTools variable string name. All the putX and getX methods are defined in JObject class. The only tricky thing is that, getObject could return null. In that case, we could check for that situation and assign object only when the return object is not null.

6. Property Related Methods in Module

The property related methods are:

JArray getProperties();

boolean updateProperty(JProperty& prop);

 

When user right click on a module, NeatTools will call the getProperties method first to get all the properties that belong to this module. Later on, if user decide to change one of the property, after property have been changed and user pressed the OK button, NeatTools will invoke updateProperty with a property parameter. The JProperty is the base class for every property. Currently, NeatTools has JIntegerProperty, JIntegerListProperty, JStringProperty, and JFileProperty. All you need is include the header files and look at the header files themself or copy the usage from other module's source code.

The default behavior is following:

JArray JModuleObj::getProperties() {

JArray properties;

properties.append(JColorProperty("moduleColor", moduleColor));

return properties;

}

 

boolean JModuleObj::updateProperty(JProperty& prop) {

if (prop.getName() == JString("moduleColor")) {

moduleColor = ((JColorProperty*)&prop)->color;

repaintView(*this);

return true;

}

return false;

}

 

All module has a moduleColor property that define the base color of module. In getProperties, we initialize a property array and append a JColorProperty instance with name of "moduleColor". When NeatTools call updateProperty, we have to check the property's getName method and make sure we are handling the right property. Then we proceed the needed operations. If we process the property, we should return true. So NeatTools knows this property has been processed. Otherwise, we return false.

In case we derived from other module, for example, JLEDObj derived from JModuleObj. The implementation is a little bit different:

JArray JLEDObj::getProperties() {

JArray properties = JModuleObj::getProperties();

properties.append(JIntegerProperty("value", value, -limit-1, limit));

properties.append(JColorProperty("color", color));

properties.append(JColorProperty("bkgnd", bkgnd));

return properties;

}

 

boolean JLEDObj::updateProperty(JProperty& prop) {

if (JModuleObj::updateProperty(prop)) return true;

if (prop.getName() == JString("value")) {

setValue(((JIntegerProperty*)&prop)->value);

return true;

} else if (prop.getName() == JString("color")) {

color = ((JColorProperty*)&prop)->color;

repaintView(*this);

return true;

} else if (prop.getName() == JString("bkgnd")) {

bkgnd = ((JColorProperty*)&prop)->color;

repaintView(*this);

return true;

}

return false;

}

 

When we derive class from other module class, in getProperties method, we usually call base class' getProperties first and get its property array. Then, we append properties that belong to the derived class. In updateProperty method, we call the base class' updateProperty method first, if it return true, it means the property is belong to the base class and the property is already been processed. In this case, we return true also. Otherwise, we proceed to check the property and use the new property value to setup module's internal state or operations needed.

7. About Polymorph Data Type

The polymorph data type is a special data type that could connection to any data type. NeatTools queries the data type by invoke inputType or outputType. When programmer dealing with polymorth data type, he will use the inputSet and outputSet to get the connected module's inputType or outputType to recursively get the current data type. But care should be taken on the cyclic situation. Basically, we use counter for each port to make sure the cyclic situation does not happen. When access method been call, we redirect it to the access of the connected module. Cyclic could happen, so programmer need to do some extra code to avoid that. There are several module use polymorph data type like JMultiplexer and JDeMultiplexer. Please reference their source code for more information.

8. Use Thread in Module

In some situations, programmer will need to handle or process something without hanging the NeatTools system. For example, in JSocketObj, after the socket and server socket is connected, how we know a socket package is arraived? We can use non-blocking socket to test if package is ready for read. But how long should we check the socket input? If we use the blocked socket to read package, it could happen that no package is coming in and your module stuck NeatTools. In this case, if we dedicate a separate thread loop to execute the socket read, even the thread is block by the socket read procedure, the NeatTools still can proceed without stopped by modules.

How we use thread in module? Lets look at the code from JTimeObj module.

JTimeObj::~JTimeObj() { close();}

 

void JTimeObj::run() {

valid = true;

updateTime();

while (valid) {

while (!isTimeChanged() && valid)

JThread::sleep(10);

updateTime();

if (!valid) break;

JThread::sleep(900);

broadcast(0);

}

}

 

void JTimeObj::reset() { valid = false;}

 

void JTimeObj::startup() {

thread = JThread(this);

thread.start();

}

 

void JTimeObj::close() {

if (!valid) return;

valid = false;

thread.waitFor();

}

 

When NeatTools prepare a new instance of module, it will first call the startup method. So, this method could be a good place to start a new thread. Here, we first initialize a new thread and assign it into thread variable. After that, we call thread.start method. If everything went well, a thread will start to running begin by invoke run method. Usually, we will have a loop inside run method. If we exit the run method, the thread stopped and removed by system. If the module's instance been remove by system, it will call the ~JTimeObj and eventually call the close method. In close method, we mark the valid to false and wait for thread to exit. Do not delete thread directly. Use waitFor method instead to avoid the memory-leaking problem. Now, their is one problem left, the valid is true to indicate their is a thread running inside JTimeObj module. But it could happen that user duplicate an active module. After duplicate operation, the new JTimeObj instance has true value in valid variable but it actually does not has a thread running. So, the reset method comes into action. When user try to duplicate module, NeatTools will try to invoke the reset method on the new module. So if we set valid to false in reset method, everything will go to be perfect.

9. Concurrency Issues when Design Module

Because NeatTools allow module use their own threads, the concurrency problems could happen. Programmer should keep it in mind that it's engine and access method could be called by other modules at the same time. For example, lets look at the following situation in engine method:

void JTNG4Obj::engine(int n, JLinkObj& link) {

switch (n) {

case IN_COM: {

JBlock _data;

link.access(JBytesData(_data));

csb.lock();

buf+=_data;

csb.unlock();

break;

}

...

}

}

 

In JTNG4Obj module, I use a JBlock variable 'buf' in it. Because, the engine method could be invoke by more than one thread at the same time, it could happen that different events execute the buf+=_data statement almost at the same time. What could happen? Information lost could happen. Or worse, NeatTools could crash. Here, I use an instance of JCriticalSection class to guard the statements from access by more than one thread. The lock method will allow only one thread enters. And unlock will turn off the critical section lock. To determine where you should use JCriticalSection to guard your critical section statements need experience and knowledge on module design. I will leave the concurrent problems to programmer themselves.

 

 

2. Bibliography

1. Nancy A. Lynch and Mark R. Tuttle. An introduction to Input / Output Automata. CWI-Quarterly, 2(3), 1989.
2. Myers, Brad A. "User Interface Software Tools", ACM Transactions on Computer-Human Interaction, 2(1): 64-103, March 1995.
3. Myers, Brad A. and Mary Beth Rosson. Survey on User Interface Programming. In Proceedings of SIGCHI'92 Human Factors in Computing Systems, May 1992.
4. Myers, Brad A. "Challenges of HCI Design and Implementation", ACM Interactions 1, 1(1994), to appear.
5. X Business Group, Inc. Interface Develop Technology. 3155 Kearney Street, Suite 160, Fremont, CA 94538. (510) 266-1075, 1944.
6. Wilson, David. "Programming with MacApp pages", Addison-Wesley Publishing Company Reading, MA, 1990.
7. Linton, Mark A., John M. Vlissides and Paul R. Calder. "Composing User Interfaces with InterViews". IEEE Computer 22(2): 8-22, February 1989.
8. Microsoft Corporation. Windows 95, Windows 98, Windows NT, Visual C++, Visual Basic, Visual J++, MFC, COM/DCOM, and Internet Explorer. PO Box 72368, Roselle IL 60172-9900. (800) 426-9400, 1997.
9. Szekely, Pedro. "Separating the User Interface from the Functionality of Application Programs". Ph.D. Thesis. Carnegie Mellon University School of Computer Science technical reports CMU-CS-88-101, January 1988.
10. Laura Lemay and Charles L. Perkins. "Teach Yourself Java in 21 Days", Sams Net, 201 West 103^rd Street, Indianapolis, Indiana 46290.
11. Richard C. Lee and William M. Tepfenhart. "UML and C++, A Practical Guide To Object-Oriented Development", AT&T Lucent Technologies, Prentice Hall, Upper Saddle River, NJ 07458.
12. Richard J. Simon, Tony Davis, John Eaton, and R. Murray Goertz. "Windows 95 Multimedia & ODBC API Bible", Waite Group Press, 200 Tamal Plaza, Corte Madera, CA 94925.
13. Charles Petzold. "Programming Windows 3.1" Third Edition, Microsoft Press, A Division of Microsoft Corporation, One Microsoft Way, Redmond, Washington 98052-6399.
14. Jon Siegel. "CORBA, Fundamentals and Programming", Wiley Computer Publishing, John Wiley & Sons, Inc. Professional, Reference and Trade Group, 605 Third Avenue, New York, NY 10158-0012
15. Oracle Corporation. "Oracle7 Server Concepts Manual", Oracle7 Server Documentation Manager, Oracle Corporation, 500 Oracle Parkway, Redwood Shores, CA 94065.
16. Matt Pietrek. "Windows Internals, The Implementation of the Windows Operating Environment", Addison-Wesley Publishing Company (800)358-4566.
17. Bradford Nichols, Dick Buttlar & Jacqueline Proulx Farrell. "Pthreads programming", O'Reilly & Associates, Inc. 101 Morris Street, Sebastopol, CA 95472-9902.
18. Nabajyoti Barkakati. "X Window System Programming, Your Complete Guide to Developing X and Motif Applications!", Sames Publishing, 201 W. 103^rd Street, Indianapolis, Indiana 46290.
19. Kurt Hensen "An Introduction to the Practical Use of Colored Petri Nets", Department of Computer Science, University of Aarhus, Ny Munkegade, Bldg. 540, DK-8000 Aarhus C, Denmark
20. Object Managements Group, http://www.omg.org/, 492 Old Connecticut Path, Framingham. MA 01701, USA. (508)820-4300.
21. Rational Software Corporation, http://www.rational.com/, 4900 Perl East Circle, Suite 106, Boulder, CO 80301, USA. (303)444-3464.
22. Advanced Visual Systems Inc. http://www.avs.com/, 300Fifth Avenue, Waltham, MA 02154, USA. (781)890-4300.
23. National Instruments Corporation. http://www.natinst.com/, 11500N. Mopac Expwy, Austin, TX 78759-3504, USA. (512)794-0100.
24. JavaSoft. http://www.javasoft.com/, (888)543-5282.
25. JavaSoft. "Java Language", "JDK1.1", "JDK1.2", "HotSpot", "JIT", "JDBC", " The Java Language Environment - A White Paper". http://www.javasoft.com/, (888)543-5282.
26. Linux Online, "LINUX Operating System". http://www.linux.org/.
27. GNU Project Web Site. "GCC C/C++ compiler". http://www.gnu.org/.
28. Institute for Interventional Informatics. "NeatTools", "Neat DOS", "TNG". http://www.pulsar.org/.
29. Netscape Corp. "Netscape Communicator". http://www.netscape.com/.
30. Advanced Micro Devices, Inc. "AMD K6-233 CPU". http://www.amd.com/.
31. POSIX Software Group. "POSIX Thread (pThread)". http://www.posix.com/.
32. Inprise Corporation. "Object Window Library", "Boland C++". http://www.borland.com/.
33. Open Software Foundation. "MOTIF", "Distributed Computing Environment ".
34. Sun Microsystem Corp. "SOLARIS", "ULTRA Workstation". http://www.sun.com/.
35. Sybase, Inc. 6475 Christie Ave. Emeryville, CA 94608. http://www.sybase.com/.
36. IBM North America, 1133 Westchester Avenue, White Plains NY 10604. "IBM PC". http://www.ibm.com/.
37. Apple Computer, Inc. 1 Infinite Loop Cupertino, CA 95014. "Macintosh", "Power PC". http://www.apple.com/, (408)996-1010.
38. Bristol Technology, Inc. "Win/U". http://www.bristol.com/.

NAME OF AUTHOR: Yuh-Jye Chang

PLACE OF BIRTH: Wen-Lin, Taiwan

DATE OF BIRTH: Feb 22, 1966

Stevens Institute of Technology, Hoboken, New Jersey

National Taiwan University, Taipei, Taiwan

Master of Computer Science, 1991, Stevens Institute of Technology

Bachelor of Mechanical Engineering, 1988, National Taiwan University

Creative Entertainment Choice, 1997

Java Cup International Winner, 1996

National Taiwan University Symphony Vice-leader, 1987

Research Engineer, Bell Labs, Lucent Technologies, 1998

Research Assistant, Northeast Parallel Architectures Center, Syracuse University, 1997

Teaching Assistant, Department of Computer Information Science, Syracuse University, 1996

Project Leader, NewTek Corp, 1992

Robot Lab Assistant, Department of Mechanical Engineering, National Taiwan University, 1987