flow/packet loss through L3 C3560, pings OK

flow/packet loss through L3 C3560, pings OK

NewsGroups | Search | Tools
 comp.dcom.sys.cisco  Post an article  get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content  add this group's latest topics to your Google content  YahooMyWeb Yahoo!  Google Google  Windows Live Favorites Windows Live  del.icio.us del.icio.us  digg digg  Add to Netscape Netscape
Subject Author Date
flow/packet loss through L3 C3560, pings OK Phil Begriffenfeldt 05-01-2008
Posted by Phil Begriffenfeldt on May 1, 2008, 12:59 am
If you were  Registered and logged in, you could reply and use other advanced thread options
I have a Catalyst 3560G that is doing L3 routing. I tried to use it as
default gateway for a web cluster, which was doing about 120mbps of
traffic, 5kpps each in and out. However, users noticed slow page loads,
broken inline images, etc.

I was able to ping all the servers from outside the 3560G with zero
packet loss in tens of thousands of 1500-byte pings. I moved the web
cluster to a C6509 (same interface config) and the issue disappeared.

Web client experience was noticably impacted, so if it were simple
packet loss, I think I would have seen it with ping. It seemed as
though the issue was related either to the type of traffic (plain http)
or flow (lots of flows).

The 3560 has a pretty vanilla config; the web cluster traffic was being
routed between a "no switchport" interface and a Vlan interface. I did
notice that the "no switchport" interface had "ip route-cache
same-interface" configured, and I'm not sure why. Also, the 3560 is
carrying about 7k external routes, but I monitor it to make sure it
doesn't hit the limit. I didn't see any clues in syslog.

Phil

Network Magic Graduation 20% off animated banner
Posted by stephen on May 1, 2008, 5:37 pm
If you were  Registered and logged in, you could reply and use other advanced thread options
> I have a Catalyst 3560G that is doing L3 routing. I tried to use it as
> default gateway for a web cluster, which was doing about 120mbps of
> traffic, 5kpps each in and out. However, users noticed slow page loads,
> broken inline images, etc.
>
> I was able to ping all the servers from outside the 3560G with zero
> packet loss in tens of thousands of 1500-byte pings. I moved the web
> cluster to a C6509 (same interface config) and the issue disappeared.

we had some issues with the 10/100 versions with buffer tuning where we had
problems with traffic bursts overwhelming the buffers, esp when you turn QoS
on as you effectively reduce the buffer pool for any 1 QoS type by 75%.

If you have several GigE connected servers contending for a congested or
rate limited port this could be an issue.

there are some commands to look at the buffers - something like
show platform port-asic statistics..... you want the drop stats for any
overloaded outbound ports.
>
> Web client experience was noticably impacted, so if it were simple
> packet loss, I think I would have seen it with ping. It seemed as
> though the issue was related either to the type of traffic (plain http)
> or flow (lots of flows).
>
> The 3560 has a pretty vanilla config; the web cluster traffic was being
> routed between a "no switchport" interface and a Vlan interface. I did
> notice that the "no switchport" interface had "ip route-cache
> same-interface" configured, and I'm not sure why. Also, the 3560 is
> carrying about 7k external routes, but I monitor it to make sure it
> doesn't hit the limit. I didn't see any clues in syslog.

you need "sdm prefer routing" in the config to handle lots of IP routes - if
not they overflow the hardware forwarding table and get dealt with in
software.
>
> Phil
--
Regards

stephen_hope@xyzworld.com - replace xyz with ntl



Posted by Phil Begriffenfeldt on May 2, 2008, 9:28 am
If you were  Registered and logged in, you could reply and use other advanced thread options
stephen wrote:
> you need "sdm prefer routing" in the config to handle lots of IP routes

Thanks for reminding me. I did set that last May (it's logged), and
then power-cycled the switch, but I do not appear to have verified "show
sdm" after the power cycle. Now I see that the switch is using
default/desktop, which could be the source of my trouble. Weird.

Posted by stephen on May 2, 2008, 3:11 pm
If you were  Registered and logged in, you could reply and use other advanced thread options
> stephen wrote:
> > you need "sdm prefer routing" in the config to handle lots of IP routes
>
> Thanks for reminding me. I did set that last May (it's logged), and
> then power-cycled the switch, but I do not appear to have verified "show
> sdm" after the power cycle. Now I see that the switch is using
> default/desktop, which could be the source of my trouble. Weird.

yes - hardware forwarding tables will fill with 1 to 2k routes.

everything that arrives after the tables fill goes in software forwarding -
so whether it is an irritation or a disaster depends on the order the routes
arrive.

Not a fun thing to trouble shoot, but it does log an "out of space"
message - shame Cisco couldnt make it obvious what it is an error about....
--
Regards

stephen_hope@xyzworld.com - replace xyz with ntl



Similar ThreadsPosted
Files on a C3560? March 1, 2006, 6:48 am
C3560 vs C3750 switches July 13, 2005, 3:31 pm
VLANs routed with C3560 and Proxy ARP August 10, 2008, 1:08 pm
I have strange routing problem with my C3560 switch. March 7, 2007, 7:20 am
Wireless pings June 3, 2008, 4:39 pm
Cisco 800 - only repond to pings from certain addresses? May 5, 2006, 3:17 am
ACL: Does "permit IP" allow ICMP traffic like pings? January 4, 2007, 12:01 pm
Re: Linksys wireless-G notebook adapter pings Cisco web each second March 23, 2008, 11:17 pm
Re: Linksys wireless-G notebook adapter pings Cisco each second March 24, 2008, 1:57 am
At a loss October 5, 2005, 3:30 am

other useful resources:
The Federal Communications Commission (FCC)
Telecommunications Industry Association
Electronic and Software Security Products and Services
International Telecommunication Union

Custom CGI Perl and PHP programming by 1-Script.com

Contact Us | Privacy Policy
The site map in XML format XML site map