This report collects together the results of two workshops held under the sponsorship of NSF Grant ANI-0135530 from Advanced Networking and Infrastructure Research Division. The first workshop on “Modeling and Simulation of Ultra Large Networks: Challenges and New Research Directions” was held on Nov. 19-20, 2001 in Tucson, AZ, USA. The second workshop on “Ultra Large Networks: New Directions in Modeling and Simulation-based Security”, was held on May 29-31, 2003 at Stevens Institute of Technology, Hoboken, NJ. Both workshops were organized by The Arizona Center for Integrative Modeling and Simulation (ACIMS) and The Society for Modeling and Simulation International (SCS). The Secure Network Systems Design Laboratory (SENDLAB) participated in the organization of the second workshop.

The purpose of the workshops was to bring together for short periods experts in the networking area to meet with counterparts in modeling and simulation. The first workshop explored ultra-large networks in general with the intent of generating a set of recommendations for new directions in research for modeling and simulation. Subsequently, a second meeting was held, with mostly different participants, which focused on the more specific problems of security in ultra-large networks, again from the perspective of research issues to be investigated using modeling and simulation.

This final report for the grant collects three reports together that emerged from the proceedings of the two workshops. Two reports, generated as results of the first meeting, are:

The third report, generated as a result of the second workshop is:

· Second Workshop on Ultra Large Networks: New Research Directions in Modeling and Simulation-based Security

In the following, we summarize the main points of the three reports.

Summary of First Report

The first report documents the definition of Ultra-large Networks and thirteen rank-ordered findings and recommendations that were formulated during the workshop and associated discussions using the Group Systems laboratory at the Center for Management of Information, University of Arizona. These statements were subsequently posted on the ACIMS web site with access restricted to workshop participants for comment. The report includes a bibliography of articles and research papers, in part supplied by the participants, in part gathered from the literature. The findings and recommendations were categorized under the following headings:

Modeling

Technologies for model validation are needed and go hand in hand with progress in acquiring realistic Internet data
Common repository of tools needed. Stop reinventing the wheel, but facilitate truly new invention.
Multiplicity, Scalability and Heterogeneity are difficult, but important issues.
ULN poses particular challenges to moving from an art, to a science, of multi-resolution/multi-aspect modeling.

Simulation

Real-time modeling and simulation (M&S) is important in itself and as a crucible for other M&S issues.
Effective models and simulators, properly packaged and annotated to be promulgated widely to support ULN research and application.

Theory

Various approaches are needed to deal with unanticipated or emergent phenomenon, including rare events.
New mathematical approaches and metrics are needed, either borrowed from other areas, or specifically developed for ULN.

Measurement

Progress in ULN depends critically on acquiring, characterizing and archiving real-world data.

Analysis

We need to understand the evolution of the Internet – other researchers such as historians, economists and biologists may be helpful.

Design and Operation

Evolutionary and ‘Blank Slate’ approaches should coexist.
Interfacing M&S with other tools is critical to deal with many aspects of ULN design and operation.
Extrapolating from the Internet of today to the ULN of tomorrow raises many questions that need examination.

Summary of Second Report

The second report was prepared in the year following the workshop and is based in part on the workshop discussions and recommendations, and in part on subsequent study of the literature as well as exchanges with workshop participants and others. In this report, some representative issues and problems are discussed that arise when the Internet is examined from the point of view of its future as an Ultra-large Network (ULN). Some issues that are in critical need of modeling and simulation for their solution are enumerated and employed as a basis to evaluate current network simulators on their ability to support ULN studies. Finding that current simulators fall short in many aspects, we developed a framework to place modeling and simulation activities within an iterative cycle for ULN development. This framework enabled us to expose the current limitations of modeling and simulation tools in supporting the iterative design cycle. The report concludes by examining the research needed to meet the challenges of ULN applications. Its conclusions are categorized under components of the iterative design cycle as follows:

Theory: Research is needed to enable M&S to provide the means to test theories of ULN behavior and to stimulate their development. Such theories may require introduction of new mathematical concepts as well as borrowing of existing concepts from existing complex systems.
Experiment: Research is needed to link development of models with experiments that can be done on the Internet itself or within testbeds developed for such purposes. Models should help translate general theories of ULN phenomena into concretely testable hypotheses enabling simulations to generate behavior comparable with experiment.
Measurement: Research is needed to enable M&S to support the development of measurement techniques – whether employed online or for research – that provide veridical and to-the-point information on the state of the network
Analysis: Research is needed to enable M&S to work seamlessly traditional analysis techniques so that for example, mathematically tractable abstractions of simulation models can be generated automatically.
Design/Operation: Research is needed to enable

New design proposals and/or system interventions to be developed and tested in "virtual" testbeds - realistic characterizations of the real Internet environment of today or the ULN of tomorrow. Such characterizations will require developement of new levels of abstraction and new models within such levels to be able to effectively and efficiently address the questions raised by proposed designs and interventions.
Families of models at various levels of abstraction to be constructed, cross-validated against realistic Internet data, and placed in common repository of models accessible to all researchers.
Efficient simulation of models using single processor, parallel and/or distributed environments and the widespread dissemination of these simulators.

Summary of Third Report

Although a broad picture of the critical role of modeling and simulation in future ULN development emerged from the Tucson workshop, there remained the need to concretize the recommendations and findings to more specific issues. Security, an issue that emerged after the events of September 11, 2001 to capture universal attention, was considered to be a timely topic for consideration. With the added help of the Secure Network Systems Design Laboratory (SENDLAB) a workshop was designed to bring together users of cyberspace networks and researchers in networking, modeling, and simulation to identify key user requirements for network security, and to translate these requirements into definitive simulation-based design approaches for future robust and secure ultra-large networks.

The participants of the second workshop proposed the goal of developing "inherently secure architecture" that scales to large networks and does not depend on the ever more frequent patches and security updates. They envisioned a framework for network security transcends the traditional end-to-end encryption paradigm and assumes the form of an engineering concept that lends itself to systematic design, scientific measurement, and quantitative assessment. This framework should enable employing modeling and simulation to address the following urgent requirements:

investigate threat scenarios that cut across the hardware, software, and algorithmic elements of complex networks.
develop a logical set of principles to automatically identify threats to different types of networks including communications, control, and sensor.
understand the difficult tradeoffs between increased security and reduced privacy
identify the requirements of future network design from the perspective of law enforcement.
study biologically inspired approaches to strengthen critical infrastructure networks.
develop automated mechanisms to generate a dynamic “weather” map of the Internet, especially focusing on the link bandwidths and their utilizations.

The workshop participants concluded that modeling and simulation is a must-use tool needed to incorporate security as a network design parameter and measure its impact on the network performance.

Conclusions

The Internet, perhaps the largest complex technological information processing system that we know, rivals the human brain as the largest (in terms of number of elements) complex information processing natural system that we know. Researchers today are only starting to get a handle on the secret workings of the brain through new observation techniques that allow more fine grained observation of activity levels in various structures. Likewise, we are only starting to get a perspective on the true complexity of devices and connections composing the Internet through maps derived from sophisticated measurements such as port scanning and pinging techniques. Observation methods such as these are likely to grow in capability but the accumulation of data is not likely to produce improved understanding of deeper laws of behavior without the capability to model and simulate such large networks. Like the brain, the Internet is an organic system, not fully capable of control by its designers, and dependent on the whims and idiosyncrasies of its (human) constituents. And both ultra-large networks, will challenge the tools that modeling and simulation has to offer. But combined, with other essential activities, such as theory development, analysis, and validation against observation, modeling and simulation provides the only feasible and scalable approach to capture the complexities of such networks and to support better interventions and designs that will improve their operation.