Implementation vs. Simulation: Evaluating a MANET Multicast Protocol

Thomas Kunz

Systems and Computer Engineering

Carleton University

http://kunz-pc.sce.carleton.ca/

tkunz@sce.carleton.ca

Motivation

Ad hoc networks: infrastructure-less, rapid deployment

Many applications for ad hoc networks require one-to-many and many-to-many communication

Multicast protocols are intended to efficiently support such communication patterns

Multicasting well researched in fixed networks (i.e., the Internet), building efficient distribution structures (typically a multicast tree)

Ad hoc networks: dynamic topology makes it harder to maintain distribution structure with low overhead

Motivation (cont.)

Previous work: studied performance of multicast and broadcast protocols in NS2, determined that broadcast protocols can support one-to-many and many-to-many communications well in a MANET

Choose BCAST as starting point, a protocol that utilizes 2-hop neighborhood information to dynamically prune re-broadcasts

However, even for “best” broadcast protocol, Packet Delivery Ratio (PDR) can be reduced when

Mobility increases

Number of multicast sender (i.e., traffic) increases

“Reliable” Protocol: Key Design Alternatives

Reliability Mechanism?

Forward Error Correction: Overhead with each packet, design often based on worst-case assumptions

Retransmissions: detect packet loss and recover

Ack-based

Nack-based

Which Protocol Layer?

Transport Layer

Routing Layer

Flow Control, Security, etc. (not considered)

Reliable BCAST

Each node keeps cache of recently transmitted packets (FIFO, small)

Each node, upon receiving packet X from sender S, checks whether it received packet X-1 from that sender

If not, broadcast retransmission request (NACK) to 1-hop neighbors

Neighbors listen to overhear other retransmission and cancels theirs

First set of experiments revealed that under high traffic load, too many NACKs were issued, flooding the network and resulting in overall worse performance (70% PDR for 10 sender scenarios)

Added feature: NACK throttle

Results: consistent improvement in PDR compared to best-effort version, at cost of slightly higher packet latency (all performance differences statistically significant at 99% confidence level).

But are simulation results true….?

Idea: compare simulation results with testbed

Implemented BCAST on Linux laptops and IEEE 802.11a/b/g Netgear wireless cards

Compare results

Static network topologies

Dynamic network topologies

Emulate node movement

Used MNE (Mobile Network Emulator from NRL)

Comparison 1: Static Network

Comparison 2: Dynamic Networks

Analysis of Dynamic Network Results

Not the same quantitative results (was expected)

However, also not the same qualitative results!

Reliable BCAST does not increase the PDR for low traffic loads (independent of the mobility rate):

Under light traffic load, the single-sender scenario has a lower PDR than the multiple-sender scenario, yet the reliable protocol version does not recover from the single-transmitter losses (as static network results show, PDR of 100% is possible)

Reliable BCAST performs worse under heavy traffic loads (many multicast sender sending lots of data)

Under heavy traffic load, the increase in the number of senders caused severe congestion, producing a significant drop in PDR for all mobility rates and protocol versions.

Conclusions and Future Work

Implemented a multicast/broadcast protocol both in a popular network simulator (NS2) and on Linux laptops

Compared performance under a range of scenarios

Quantitative and qualitative differences

Quantitative differences expected

Qualitative differences not expected and cause for concern

Future work:

Include additional simulators (GloMoSim) and tests without MNE

Analyze observed differences closer

Isolate the effect of MNE

Additionally, work on core protocol (congestion control, security)


	Thomas Kunz
	Systems and Computer Engineering
	Carleton University
	http://kunz-pc.sce.carleton.ca/
	tkunz@sce.carleton.ca


	Ad hoc networks: infrastructure-less, rapid deployment
	Many applications for ad hoc networks require one-to-many and many-to-many communication
	Multicast protocols are intended to efficiently support such communication patterns
	Multicasting well researched in fixed networks (i.e., the Internet), building efficient distribution structures (typically a multicast tree)
	Ad hoc networks: dynamic topology makes it harder to maintain distribution structure with low overhead


	Previous work: studied performance of multicast and broadcast protocols in NS2, determined that broadcast protocols can support one-to-many and many-to-many communications well in a MANET
		Choose BCAST as starting point, a protocol that utilizes 2-hop neighborhood information to dynamically prune re-broadcasts
	However, even for “best” broadcast protocol, Packet Delivery Ratio (PDR) can be reduced when
		Mobility increases
		Number of multicast sender (i.e., traffic) increases


Reliability Mechanism?
	Forward Error Correction: Overhead with each packet, design often based on worst-case assumptions
	Retransmissions: detect packet loss and recover
		Ack-based
		Nack-based
Which Protocol Layer?
	Transport Layer
	Routing Layer
Flow Control, Security, etc. (not considered)


	Each node keeps cache of recently transmitted packets (FIFO, small)
	Each node, upon receiving packet X from sender S, checks whether it received packet X-1 from that sender
		If not, broadcast retransmission request (NACK) to 1-hop neighbors
		Neighbors listen to overhear other retransmission and cancels theirs
	First set of experiments revealed that under high traffic load, too many NACKs were issued, flooding the network and resulting in overall worse performance (70% PDR for 10 sender scenarios)
	Added feature: NACK throttle
	Results: consistent improvement in PDR compared to best-effort version, at cost of slightly higher packet latency (all performance differences statistically significant at 99% confidence level).


Idea: compare simulation results with testbed
Implemented BCAST on Linux laptops and IEEE 802.11a/b/g Netgear wireless cards
Compare results
	Static network topologies
	Dynamic network topologies
		Emulate node movement
		Used MNE (Mobile Network Emulator from NRL)


	Not the same quantitative results (was expected)
	However, also not the same qualitative results!
	Reliable BCAST does not increase the PDR for low traffic loads (independent of the mobility rate):
		Under light traffic load, the single-sender scenario has a lower PDR than the multiple-sender scenario, yet the reliable protocol version does not recover from the single-transmitter losses (as static network results show, PDR of 100% is possible)
	Reliable BCAST performs worse under heavy traffic loads (many multicast sender sending lots of data)
		Under heavy traffic load, the increase in the number of senders caused severe congestion, producing a significant drop in PDR for all mobility rates and protocol versions.


	Implemented a multicast/broadcast protocol both in a popular network simulator (NS2) and on Linux laptops
	Compared performance under a range of scenarios
	Quantitative and qualitative differences
		Quantitative differences expected
		Qualitative differences not expected and cause for concern
	Future work:
		Include additional simulators (GloMoSim) and tests without MNE
		Analyze observed differences closer
		Isolate the effect of MNE
	Additionally, work on core protocol (congestion control, security)