Re: supercomputing

From: Andre Fourie <fourie_at_email.domain.hidden>
Date: Thu Aug 15 21:49 MET 1996

Gerry v Dooren wrote:

>Now we are on the subject, does anybody have any info on a parallel
>implementation of NEC?The SP2 machine we have here has 16 nodes, of which 8 ca
>n be used in a parallel configuration. At the moment I am using a node as
>stand-alone system, because of the fact that I don't have any dedicated,
>parallel matrix factorisation and solution algorithms.

>Any info is appreciated.

We have been using a locally redesigned version of NEC2 running in
parallel for some years. We have called the program SuperNEC and in
addition to the fully "parallelized" numerical routines, we have also
added an iterative solution technique and a Matlab based user interface.

Parallel execution is currently achieved using PVM public domain software
which allows us to execute in parallel on connected workstations. By
sheer coincidence my colleague, Derek Nitch, is currently porting SuperNEC
to a 28 node SP2 system while doing some work in the UK. All numerically
intensive routines were rewritten to execute in parallel (matrix fill,
solve, radiation pattern, fields etc). Execution efficiencies of close
to 90% have been achieved for problems in the order of 5000 segments.
Memory is also managed "in parallel" with the result that the largest
matrix can use the combined memory of all machines on the network.

I am including a brief overview on the parallel SuperNEC development
history and also a summary of the SuperNEC program which I prepared
earlier for Ed Miller's Aces Software Sourcebook.

NEC2 parallel: Overview of efforts at Dept. Elec. Eng. at the
=============================================================
University of the Witwatersrand, South Africa.
==============================================

A number of years ago Derek Nitch implemented a parallel version of
NEC2 using a mixture of FORTRAN and OCCAM code which he implemented
and tested on INMOS T800 transputers. This effort was reported on in:

DC Nitch and APC Fourie, 'Adapting NEC to run in parallel on a
network of transputers'. In SAIEE Third Joint Symposium on Antennas
and Propagation and Microwave Theory and Techniques, pages 31-38.
Somerset West, August 1990. An M.Sc. dissertation on this topic was
also submitted by Derek Nitch in June 1990 at this university.

Parallel efficiencies in the order of 90% and higher were obtained
for both matrix filling and factoring.

We found the FORTRAN code difficult to generalize to different
parallel platforms and NEC2 theory was redesigned using the Object
Oriented Paradigm and coded using the C++ language. This produced a
program, which we call SuperNEC, which can be ported to different
platforms with ease and in general provided a program very suitable
to modifications. Some of this work is reported on in:

DC Nitch and APC Fourie: 'A redesign of NEC2 using the object
oriented paradigm' in APS International Symposium, Seattle, USA, pp
1150-1153, 1994.

as well as a Ph. D. thesis containing all the detail:

DC Nitch 'A serial and parallel design of NEC2 to demonstrate the
advantages of the objecto oriented paradigm in comparision with the
procedural paradigm'. PhD thesis, University of the Witwatersrand,
Johannesburg, December 1992.

To illustrate one of the benefits of the object oriented SuperNEC
program it was once again modified for parallel execution on T800's
in a period of two weeks as opposed to the 6 months required for the
previous FORTRAN version (excluding all the learning curves i.e.
referring to coding time only). This was reported on in

DC Nitch and APC Fourie, 'Parallel implementation of NEC'. Applied
Computational Electromagnetics Society (ACES) Journal, 9(1) p51-57,
March 1994 as well as the PhD thesis cited above.

Some iterative solution techniques and other modifications
have been added to the SuperNEC implementation, some of these
reported on in:

APC Fourie, and DC Nitch, 'A fast sparse interative method (SIM) for
method of moments' as well as 'Comparing the sparse iterative method
with the banded Jacobi and conjugate gradient techniques.' Both papers
presented at APS Symposium in Seattle, USA, 1994. pp1146-1149 and pp
1181-1184 respectively.

Recently the SuperNEC program has been adapted to run in parallel on
workstations connected with a TCPIP LAN using a public domain program
for inter machine communication called PVM (Parallel Virtual
Machine). This parallel implementation is the most attractive to date
with the following features:

- Distribution of computational effort as well as memory making use
of workstations available on a local area network.

- Different machines can be used (we have used networks with a
mixture of IBM R6000's and SUN Sparc 10's and 2's). (Load balancing
is possible, but not yet implemented)

- All important numerical components are in parallel in terms of
memory as well as execution (filling, iterative as well as LU
solving, near field and radiation pattern calculations, symmetry
computations etc).

We believe the PVM approach to be very attractive, since it is a move
away from dedicated parallel architectures and allows the user to
make use of available, general purpose workstations on the LAN.

                       THE ACES CEM SOFTWARE SOURCEBOOK

        Name of code and organization owning/developing it

SuperNEC
developed by:
EM Simulations (Pty) Ltd.

        Authors

DC Nitch
APC Fourie
EK Miller
AR Clark
OG Givati
D Redelinghuys

        Category (e.g., modeling, user interface, numerical-analysis
                algorithms, etc.)

Currently a wire based method of moments (MoM) program with a
Matlab based input and output interface.

        Purpose

Frequency domain analysis of antennas and structures by solving for the
currents given a specified excitation. Secondary parameters such as near
fields, radiation patterns, impedance, radiation efficiency, antenna-to-
antenna coupling and other related parameters computed from structure
currents.

        Language

The numerical engine is written using C++ while parts of the interface is
written using Matlab script language and C++.

        Computers on which code runs

The numerical engine has been compiled to run on IBM PC machines but is mainly
run on a variety of UNIX based workstations (Our demand up to now have required
compilation for SUN, IBM and Convex computers).

The is mainly used on Windows95 machines, but it should in principle be
possible to run on many Unix workstations supported by Matlab.

        Availability/contact (cost, address, phone & FAX number, e-mail address)

We hope to make Beta versions of the PC version of the program available to
interested users early in 1997. This will be a fully documented, stand alone
product for use on IBM compatible computers running Windows95.

Currently a parallel (PVM based) prototype version has been used by ourselves to
run large VHF/UHF aircraft antenna problems. This version, which executes in
parallel on workstations linked via a local area network, is clearly more
powerful in terms of potential problem size (limited by combined memory and
computational speed of network), but will have to be custom installed for
interested users. A certain amount of training and support will also be required
in order to use the program effectively.

The cost of the custom installed, parallel version of SuperNEC will depend
from case to case and interested users should contact us in this regard. We have
not yet finalised any costs for the more limited PC based version which will be
released in 1997.

Contact:

Dr APC Fourie

mail: Dept Elec Eng, Wits, 2050, South Africa
email: fourie_at_odie.ee.wits.ac.za
tel: Intl + 27 11 716 5386
fax: Intl + 27 11 339 4610

        Description

SuperNEC development commenced about 7 years ago and a brief historical
overview of the development is provided below:

A number of years ago Derek Nitch implemented a parallel version of
NEC2 using a mixture of FORTRAN and OCCAM code which he implemented
and tested on INMOS T800 transputers. This effort was reported on in
 [1] (see references).

Parallel efficiencies in the order of 90% and higher were obtained
for both matrix filling and factoring.

We found the FORTRAN code difficult to generalize to different
parallel platforms and NEC2 theory was redesigned using the Object
Oriented Paradigm and coded using the C++ language. This produced a
program, which we call SuperNEC, which can be ported to different
platforms with ease and in general provided a program very suitable
to modifications. Some of this work is reported on in [2] and [3].

To illustrate one of the benefits of the object oriented SuperNEC
program it was once again modified for parallel execution on T800's
in a period of two weeks as opposed to the 6 months required for the
previous FORTRAN version (excluding all the learning curves i.e.
referring to coding time only). This was reported on in [4]

Some iterative solution techniques and other modifications
have been added to the SuperNEC implementation, some of these
reported on in [5] and [6].

Recently the SuperNEC program has been adapted to run in parallel on
workstations connected with a TCPIP LAN using a public domain program
for inter machine communication called PVM (Parallel Virtual
Machine). This parallel implementation is the most attractive to date
with the following features:

- Distribution of computational effort as well as memory making use
of workstations available on a local area network.

- Different machines can be used (we have used networks with a
mixture of IBM R6000's and SUN Sparc 10's and 2's). (Load balancing
is possible, but not yet implemented)

- All important numerical components are in parallel in terms of
memory as well as execution (filling, iterative as well as LU
solving, near field and radiation pattern calculations, symmetry
computations etc).

We believe the PVM approach to be very attractive, since it is a move
away from dedicated parallel architectures and allows the user to
make use of available, general purpose workstations on the LAN.

One of the more innovative features of the SuperNEC interface is the use of
"primitives" to construct problems. Primitives are essentially Matlab functions
used to define higher level geometries in terms of a small number of parameters.
The relevant primitive function then generates the wire representations. Current
primitives include: Yagi's, LPDA's, circular plates, spheres and ellipsoids,
rectangular plates, rectangular boxes, curve extrusions, cones and a few other
shapes and antenna types. This will firstly allow a user to construct problems
much easier and, in addition, users familiar with Matlab can add/edit primitives
to extend their library of available basic geometries.

        Limitations and approximations

The program makes use of dynamic data structures and hence there is no
program limitation on segments, connections etc. Memory and computer
time, however, ultimately limits problem size. With our rather modest
computational resources we have run problems up to about 10000 segments.

The SuperNEC program supports most of the features provided by the
original NEC2 program. Some exceptions are patches, which were
omitted from SuperNEC due to their rather limited usefulness. NEC2
patches are only applicable to full closed surfaces and most users
find wire grid representation of surfaces more useful. The NGF option
in NEC2 has also not been implemented in SuperNEC.

        Representative geometries/problems

Geometries: Simple wire antennas like Yagis, wire biconicals,
dipoles, fan antennas, helices and many auther examples
Surfaces are normally handled using wire grids obeying well
known constraines and guidelines. Specific examples of aircraft
probles to which the program have been applied to is listed below

a) a Mirage fighter aircraft: We simulated the performance of two
antennas (118MHz - 420MHz) and obtained radiation patterns and
coupling between the antennas. We also performed scale model
measurements on a 10:1 copper scale model of the aircraft in an
anechoic chamber to validate the SuperNEC results for three principle
plane pattern cuts (azimuth, pitch and roll patterns). SuperNEC models
was about 10000 segments using 1 plane of symmetry.

b) A Puma helicopter: Same study as above except for the frequency
range 30MHz-420MHz using models of about 6000 segments with no
symmetry. We used 30:1, 20:1 and 10:1 copper scale models validate
the SuperNEC results by measurement in a anechoic chamber compact range.

c) A Lockheed C-130 Hercules transport aircraft: Same study as a)
except for a frequency range of 2-420MHz. Models was up to 12 000
segments with one plane of symmetry. We used 20:1, 30:1 and 72:1
copper scale models to do measurements on in order to validate
results from SuperNEC and obtain detailed coupling between communication
and navigation antennas on the aircraft.

d) The Rooivalk Attack helicopter

Freq range 2-420MHz. Once again used scale model verification of
radiation patterns. Used SuperNEC to predict HF near-fields,
3D radiation patterns, VHF/UHF antenna-to-antenna coupling.

e) The Dakota (DC-3) transport aircraft

Simulated between 30 and 420 MHz. Radiation patterns and predicted
receive signals as above. Some additional validation of simulated
results were performed in this instance, however. We performed actual
measurements of the received signal strength from an aircraft in
flight at a specially equipped aircraft test range in South Africa.
These measured results were compared to the Antenna System Evaluation
Package (ASEP) predicted signal strengths (which uses the SuperNEC 3D
radiation pattern together with a propagation model). We achieved mean
absolute errors of less than 3 dB between measured and predicted
received powers for flights ranging over 70 nautical miles
and altitudes ranging from 300 ft to 10 0000 ft. Some of these
results (together with similar measurements on the Pilatus
trainer) will be presented at the ANTEM-96 conference
in Quebec [9] and [10].

f) The Pilatus trainer aircraft

Frequency range 100-150 MHz. Outputs as above.

        Storage and timing/operation count for representative geometries

The matrix storage required is 8 * N^2 bytes for single precision operation
where N is the number of wire segments used. Computer time becomes fairly
difficult to quote for the parallel workstation solutions (used in all of the
aircraft related studies mentioned above). Approximately 3 hours for a 5000
segment problem (including 3D radiation patterns) when using 3 SUN10
workstations and 2 IBM R6000 machines linked together (each with 64 MByte
memory).

        Extent/kinds of validation

The validation performed was listed together with the problems above for
aircraft related studies. Much of this was performed using scale model
measurements in an anechoic compact range facility. Some actual received
signal strength measurements were performed on aircraft in flight and this
was correlated with predictions of received signal strength using a
propagation prediction package (ASEP). References [5] - [11] provides
published results of some of the measured verification studies performed.
Comparing measured and computed results in [9]-[11] with earlier
verification studies [7] and [8] indicates, perhaps not so surprisingly,
that reducing measurement errors cause the biggest improvement in
these studies, rather than improvements in the simulation tools and
numerical models.

        References

[1] DC Nitch and APC Fourie, 'Adapting NEC to run in parallel on a
network of transputers'. In SAIEE Third Joint Symposium on Antennas
and Propagation and Microwave Theory and Techniques, pages 31-38.
Somerset West, August 1990. An M.Sc. dissertation on this topic was
also submitted by Derek Nitch in June 1990 at this university.

[2] DC Nitch and APC Fourie: 'A redesign of NEC2 using the object
oriented paradigm' in APS International Symposium, Seattle, USA, pp
1150-1153, 1994.

[3] DC Nitch 'A serial and parallel design of NEC2 to demonstrate the
advantages of the objecto oriented paradigm in comparision with the
procedural paradigm'. PhD thesis, University of the Witwatersrand,
Johannesburg, December 1992.

[4] DC Nitch and APC Fourie, 'Parallel implementation of NEC'. Applied
Computational Electromagnetics Society (ACES) Journal, 9(1) p51-57,
March 1994.

[5,6] APC Fourie, and DC Nitch, 'A fast sparse interative method (SIM) for
method of moments' as well as 'Comparing the sparse iterative method
with the banded Jacobi and conjugate gradient techniques.' Both papers
presented at APS Symposium in Seattle, USA, 1994. pp1146-1149 and pp
1181-1184 respectively.

[7] O Givati and APC Fourie, "Radiation Patterns of Antennas Mounted
on a Modified Mirage Aircraft", Proceedings of the IEEE Antennas
and Propagation Society International Symposium, Seattle, June 1994,
pp 1158-1161

[8] O Givati and APC Fourie, 'The NEC2 Radiation Patterns of Under-
segmented Wire Grid Models of a Fighter Aircraft Compared to Measure-
mets' Applied Computational Electromagnetics Society Journal, vol 10,
no 1, pp 5-14, ISSN 054-4887, March 1995.

[9] APC Fourie, O Givati, AR Clark and N Pascoa 'Measured and Computed
Results of Air-to-Ground communication performance of 2 aircraft at
VHF/UHF frequencies' ANTEM-96 Symposium, Quebec, Canada, August 1996.

[10] J Dresel, APC Fourie, O Givati and AR Clark, 'The Modelling of
Lossy Surfaces using a Wire Method-of-Moments Code' ANTEM-96 Symposium,
Quebec, Canada, August 1996.

[11] O Givati, APC Fourie and J Dresel 'Radiation Patterns of Antennas
Mounted on An Attack Helicopter' 12th Annual Review of Progress in
Applied Computational Electromagnetics Conference Proceedings, pp
802-817, March 1996.

e-mail: fourie_at_odie.ee.wits.ac.za
Tel: intl + 27 11 716 5386
Fax: intl + 27 11 339 4610
Address: Dr. APC Fourie, Dept Elec Eng, PO WITS, 2050, South Africa
Received on Thu Aug 15 1996 - 16:49:00 EDT

This archive was generated by hypermail 2.2.0 : Sat Oct 02 2010 - 00:10:37 EDT