Object-oriented frameworks like the solutions discussed in the previous chapter are difficult to evaluate.
One initial difficulty is to understand the intended domain of the framework and its applicability to the application under construction [BMMB97].
The frameworks which are relevant for this thesis are known as application frameworks [GB01]. Their purpose is to provide all the domain-independent functionality needed in an application. The evaluated solutions extend another application framework, the .NET Framework. In comparison to the .NET Framework they provide additional functionality for more specific domains:
In the evaluation, the Composite UI Application Block is always used with the extensions provided by the Smart Client Software Factory (Chapter 5.3).
The evaluation of the applicability of these frameworks for the Test Suite application is done by checking the fulfillment of the requirements. This is dealt in the next chapter. The applicability is the most important part in this evaluation. Only suitable frameworks are considered in the next two evaluation parts which are about further quality issues (chapter 6.3) and strategic aspects (chapter 6.4).
In all three evaluation parts the appraisal is done by using three grades:
(+) … The requirement is completely fulfilled(o) … The requirement is partly fulfilled.(-) … The requirement is not fulfilled.This chapter presents the first and most important evaluation part. It checks the fulfillment of the requirements which are defined in chapter 2. Table 2 shows a summary of this evaluation which is discussed afterwards.
| Requirement | CAB/SCSF May 2007 | Spring .NET Version 1.1 | SharpDevelop Version 2.1 |
|---|---|---|---|
| Runtime platform:
Minimum .NET Framework version |
(+) 2.0 |
(+) 1.1 |
(+) 2.0 |
| Open source and programmed in C# | (+) | (+) | (+) |
| Test modules | |||
| Define test modules | (+) | (+) | (+) |
| External configuration | (+) | (o) | (+) |
| Loose coupling | (+) | (o) | (-) |
| Lazy loading of modules | (o) | (+) | (+) |
| Modules deployment | (o) | (o) | (+) |
| GUI integration | |||
| Support for GUI extension | (+) | (-) | (o) |
| Command service | (+) | (-) | (o) |
| Loosely coupled events | (+) | (o) | (-) |
Table 2: Checking the fulfillment of the requirements which are defined in chapter 2.
All solutions run on the .NET Framework 2.0 as it is specified in the requirements.
The source code of the three frameworks is available and the used language is C#.
The .NET assemblies are an ideal candidate for representing the
test modules. Assemblies are the basic unit for versioning, security, and
deployment. Hence, they fulfill the requirements. An assembly can physically
be a standalone application (.exe) or a class library (.dll)
[Löwy05, p. 23]. In the case of a test module the choice would be a class
library. By using assemblies the requirement is already dealt by the .NET
Framework. Therefore, the investigated frameworks do not have to provide an
own solution for defining the test modules.
CAB extends the .NET assemblies by the introduction of the abstract
ModuleInit class. A concrete implementation of this class is used to
initialize the module. During module loading this concrete class is searched
by reflection and initialized through the framework.
Spring .NET does not extend the functionality of the .NET assemblies.
Because the .NET Framework already fulfills this requirement, Spring .NET
gets full grade too.
In SharpDevelop a module is defined by the .addin xml file. This file
refers to a .NET assembly by using the Identity tag. The
.addin file extends
the meta-data of the assembly with additional information.
The configuration for the module loading has to be in an external
human-readable file. Changes of the configuration need to be done without
recompiling the application. All investigated solutions use an adoption of
the Plugin pattern to fulfill this requirement. A short introduction of the
Plugin pattern can be found in
chapter 3.
The module loader of CAB can be configured via a single XML file. This
file contains the module assemblies and the dependencies between them. The
information about dependencies assures that the modules are loaded in the
correct order.
Spring .NET does not support the loading of modules in the same way as
the other solutions do. It can configure the wiring of single components in
XML files. For the reason that every module can contain many components the
configuration gets extensive. Moreover, the one who is configuring the
application needs in-depth knowledge about the dependencies of each
component in the module. A way to reduce the amount of configuration is to
use the autowiring function of Spring .NET. The setter injection with
activated autowiring is limited because it is not possible to define that
some properties of an object need to be injected and others not. Autowiring
only saves the writing of the dependency information. The components still
have to be defined in the configuration file. That is the reason why Spring
.NET does not get full mark for this requirement.
SharpDevelop scans special directories for .addin files and interprets
them. The .addin files can define dependencies to other modules in a similar
way as in CAB. The main difference to CAB is that in SharpDevelop every
module has its own configuration file whereas in CAB a central configuration
file is used.
All three frameworks ship XML schema files (XSD) for the configuration.
This simplifies the writing and editing of the configuration in an XML
schema aware editor. The requirements of this thesis also specify the
configuration via command line arguments. The frameworks do not provide
direct support for this requirement but the application can extend them to
provide this functionality. A possible solution with the CAB framework can
be seen in chapter 7.5.
Loose coupling between the modules is essential for fulfilling the
requirements which are defined in chapter 2. It allows the developing of
modules by different teams. Furthermore the modules can be isolated for
testing. This simplifies the test procedure since the dependent components
can be replaced by mock objects. Loose coupling can be achieved by
programming to an interface (see
chapter 3). Nevertheless, the loosely
coupled modules have to work together in a coherent application. This means
that the components of the modules have to be weird up.
CAB and Spring .NET support the wiring by the Dependency Injection and
the Service Locator implementation. In both frameworks the Dependency
Injection and Service Locator implementation work hand in hand together and
thus, they can even be mixed in one application. The Service Locator
implementation in Spring .NET has one drawback. It does not allow the
replacement of a service instance during runtime.
SharpDevelop had a Service Locator implementation called ServiceManager
in a previous version [HKS03, p. 109], but the current version 2.1 has
replaced the ServiceManager with static service classes. This step
simplifies the service usage but it does not allow the replacement of the
services any more. The isolating of components during a test is impossible
without replacing the dependent services. A solution for implementing the
Service Locator on top of the add-in tree is demonstrated in
chapter 5.5.
The framework still gets the grade (-) because it does not support loose
coupling by itself.
The Test Suite is an extensive application. For keeping the start-up time
at a minimum, the modules have to be lazy loaded. This strategy handles the
resources in a smart way, as only the needed ones are allocated. If a user
does not use some modules during work these modules will never be loaded.
Therefore, unused modules do not waste any resources.
The Composite UI Application Block is not able to load the modules on
demand. This is due to the fact that the application integration is done in
the module initializing code. However, CAB investigates only the modules via
reflection and instantiates the subclass of ModuleInit. The services that
the modules provide can be loaded on demand with the
ServiceCollection.AddOnDemand method.
Spring .NET does not support modules in a special way. Therefore, it has
no need to load the assemblies at the start-up process. The components,
which are needed by other components, are instantiated on demand [Spring07,
p. 29]. If more components have to use the same instance, the dependent
component can be defined as singleton. By default, singletons are
instantiated during the start-up sequence of the container. The lazy-init
attribute allows delaying the creation until the component requested for is
the first time [Spring07, p. 18].
SharpDevelop defines the extension points in the .addin configuration
files. These configuration files are read at application start-up only. The
loading of the add-in assemblies is delayed until one of its extension
points is accessed [Grunwald06a].
Deployment is a topic that is well supported by the .NET Framework. .NET
provides version control for the modules and allows side-by-side execution
of different module versions in the same process.
The Smart Client Software Factory includes help topics and a reference
application for using ClickOnce deployment. ClickOnce simplifies the
deployment tasks for the end user and the manufacturer but it has its
limitations.
(…) if a program needs to carry out privileged operations that could affect other applications or data on the target machine, such as performing unrestricted file access or accessing the registry, then it may not be suitable for deployment using ClickOnce [Noyes04].
The Test Suite requires unrestricted access to the file system and it has
to install native components like system drivers for the test devices.
Therefore, ClickOnce is not an option.
Spring .NET does not provide any special deployment features.
SharpDevelop simplifies the deployment of add-ins since the add-in files
only need to be copied into one of the specified directories. In contrast to
CAB and Spring .NET it is not necessary to modify a configuration file for
installing and uninstalling an add-in. A prefabricated add-in named AddIn
Manager allows the end user to control the add-ins. A limitation of the
AddIn Manager is that the add-ins can be installed into the user profile
directory only. This issue is caused by a security restriction of the
operating system because it cannot be guaranteed that the end user has write
access in the application directory.
It is required to do the deployment tasks without restarting the
application which is not supported by all three solutions. The reason is
that the .NET Framework cannot unload .NET modules or assemblies. However,
it is possible to load every module in a different application domain. All
application domains except of the default one can be unloaded by the .NET
Framework [Löwy05, p. 322]. The drawback of this strategy is that the
modules have to communicate through remoting with each other. Solutions,
which are using this strategy, are the System.Addin namespace introduced in
the .NET Framework 3.5 (Chapter 8.2) and the CAP .NET project [Dhungana06].
If this requirement is not fulfilled, it is acceptable since it is just
defined as a nice-to-have requirement.
The test modules have to be integrated into the Test Suite user
interface. For example, a test module needs to add a new menu item in the
menu bar of the Test Suite. The challenge is to create the extension without
having a dependency on a concrete UI technology. Furthermore, a test module
also needs also the possibility to define own GUI extensions for other
modules.
CAB provides a flexible mechanism to extend the user interface. This
mechanism consists of two parts which are integrated into the WorkItem. The
first part is the Workspace. It is used for hosting UI controls of other
modules. The second part is the UIExtensionSite. The extension site allows
the extension of exposed UI elements. Every module can use these parts to
provide own GUI extensions. Both parts are independent of the UI technology.
The Composite UI Application Block supports Windows Forms controls and
allows the hosting of WPF controls in the Workspaces. If other requirements
occur, the framework can be extended.
The Spring .NET framework does not support the building of Windows-based
applications out of the box. In version 1.1 the support is limited on
Web-based applications which are using ASP .NET. However, the framework can
be extended with the required functionality.
SharpDevelop provides GUI extensions through the .addin configuration
files. This extension mechanism is independent of the UI technology. The
core supports the handling of Windows Forms controls. If other technologies
must be used, a rewriting of the core is necessary. The rewriting of code is
not the best strategy to extend the functionality. After modifying the core
it has to be accurately tested to assure that no side-effects occur. An
advantage of SharpDevelop is that every module can register its own GUI
extensions by defining a new AddInTree path. A reusable mechanism for
hosting of UI elements like the Workspace of CAB is missing. Instead, SharpDevelop uses the specific
WorkspaceSingleton class to host the Windows
Forms controls.
An implementation of the Command design pattern [GHJV95, p. 233] is
necessary for the Test Suite. It is required to decouple the UI elements
from the command handlers.
The Composite UI Application Block contains a command service. It is
managed by the WorkItem container. It is possible that different UI elements
can register themselves as command invoker to the same command as required.
An adequate CommandAdapter is necessary for registering a UI element. If a
UI element type is not known by the framework, a new adapter can be
registered in the ICommandAdapterMapService. Furthermore, the command
supports the notification of more than one command handler. The defining of
a command handler is simple because the CommandHandler attribute just needs
to be attached to the method.
Spring .NET does not have a command implementation for user interface
elements.
SharpDevelop provides a command implementation. The commands are defined
in the .addin file as an attribute of the associated UI element. Different
UI elements can use the same command class. The command class has to
implement the ICommand interface. From this it follows that the command
class is already the command handler. It is not possible that a second
command handler can handle the same command. Another drawback is that the
command implementation is not able to handle the command state. For example,
the state is responsible for deactivating all associated UI elements if the
command cannot be executed. In SharpDevelop this is done by Conditions.
Nevertheless, the Conditions are associated directly to the UI elements
instead of associating to the commands. If more UI elements do the same task
the Conditions have to be applied to all of them. This means code
duplication in the .addin file.
1 ...
2
3 <Condition name = "ActiveWindowState" windowstate="Dirty,Untitled"
4 nowindowstate="ViewOnly" action="Disable">
5 <ToolbarItem id = "Save"
6 ...
7
8 <Condition name = "ActiveWindowState" windowstate="Dirty,Untitled"
9 nowindowstate="ViewOnly" action="Disable">
10 <MenuItem id = "Save"
11 ...
12
13 ...
Listing 11: An extract of the ICSharpCode.SharpDevelop.addin
file that shows code duplication.
Listing 11 shows the save ToolbarItem and the save
MenuItem. Both items require the same condition statement since the
items have the identical meaning. In the SharpDevelop .addin
file the condition statement is duplicated.
The framework has to support loosely coupled events for communication
between the modules. Hence, two objects can register themselves as publisher
and subscriber without knowing each other.
CAB supports the loosely coupled events even on dependency injection level.
The publishers have to define their event declaration with the
EventPublication attribute. The subscribers define the event handler
method with the EventSubscription attribute. The attributes use
a string as identifier for the event topic. The event publisher and
subscribers are wired together by the framework during the object creation.
Spring .NET has a built in support for loosely coupled events too. The
objects have to register themselves via the IEventRegistry
interface at a central registry as publisher or subscriber. It is possible
to create own event registries but it is more common to use the central
event registry which is provided by the IApplicationContext. A
Dependency Injection style of event wiring is not supported. Nevertheless,
the main drawback is that a subscriber can wire itself to a specific event
only if a unique delegate type is used for the event. Otherwise, the
subscriber has to handle all events that match with its methods signatures.
The only filter that can be applied during registration of a subscriber is
the publisher type. Though, this can be a problem because the subscriber
needs a reference to the publisher for applying the filter.
SharpDevelop does not support loosely coupled events at all.