Low FTP transfer rate when enabling Jumbo frames.

Low FTP transfer rate when enabling Jumbo frames.
NewsGroups | Search | Tools
User Password
Register Now! Lost Password?
 comp.dcom.lans.ethernet  Post an article  get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content  add this group's latest topics to your Google content  YahooMyWeb Yahoo!  Google Google  Windows Live Favorites Windows Live  del.icio.us del.icio.us  digg digg  Add to Netscape Netscape
Subject Author Date
Low FTP transfer rate when enabling Jumbo frames. fj 07-14-2006
Posted by fj on July 14, 2006, 5:46 am
If you were  Registered and logged in, you could reply and use other advanced thread options


Hi

I have a problem with FTP and Jumbo frames, and would like to hear if
anybody else has experienced the same problem. The problem is that when
I enables jumbo frames support the ftp transfer rate becomes very low
compared with non-jumbo frames ftp transferring. The actually numbers
are :

Non-jumbo frames ftp transfer rate : 16.5 MB/s.
Jumbo frames ftp transfer rate : 30 kB/s.

When I do a simple drag-and-drop of a file between the 2 computers the
transfer time is the same regardless of enabling jumbo frames or not.
The problem isn't a switch problem, since I see the same thing using
a "crossed cable".

Is there anybody there have seen the same, or is there a configuration
I have forgot to setup ?.

Regards

Flemming

P.S. My operating system is windows 2000, and I am using the
"build-in" ftp server. The FTP client is "core FTP".


Pure Networks
Posted by Rick Jones on July 14, 2006, 1:46 pm
If you were  Registered and logged in, you could reply and use other advanced thread options


> I have a problem with FTP and Jumbo frames, and would like to hear
> if anybody else has experienced the same problem. The problem is
> that when I enables jumbo frames support the ftp transfer rate
> becomes very low compared with non-jumbo frames ftp
> transferring. The actually numbers are :

> Non-jumbo frames ftp transfer rate : 16.5 MB/s.
> Jumbo frames ftp transfer rate : 30 kB/s.

> When I do a simple drag-and-drop of a file between the 2 computers the
> transfer time is the same regardless of enabling jumbo frames or not.
> The problem isn't a switch problem, since I see the same thing using
> a "crossed cable".

> Is there anybody there have seen the same, or is there a configuration
> I have forgot to setup ?.

Some thoughs:

*) does _everything_ between the two systems (inclusive) support
JumboFrame?

*) what is the TCP window size being used for the FTP data connection
(you may need to do some packet tracing to find that)

*) are there any TCP retransmissions (however one sees that in windows)

*) IIRC there are times when Windows TCP will decide to do "direct
I/O" (as it were) and hold-onto the user's buffer for DMA. Depending
on the call the user made that may block the user until the data is
ACKed by the remote, effectively turning the sliding window of TCP
into stop-start.

rick jones

--
No need to believe in either side, or any side. There is no cause.
There's only yourself. The belief is in your own precision. - Jobert
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Posted by fj on July 17, 2006, 11:08 am
If you were  Registered and logged in, you could reply and use other advanced thread options


Hi Rick

Thanks for the feedback.

*) Yes, all the equipment supports jumbo frames.

*) The TCP windows size is 64K.

*) I am not quite sure about what you are saying about "user's buffer
for DMA". Could you elaborate on that?

Another thing I have discovered is that I do not have any problem with
TCP traffic when the MTU size is 4088 bytes, and I do not have any
problems with UDP traffic ( MTU = 9014 bytes ).

I have tried to capture the traffic with ethereal, and there I can see
that it takes 0.2 Sec to get an acknowledge from the receiving part
when MTU=9014 bytes. Traffic with MTU=1518 bytes gives acknowledge
within a few msec, so I expect that this is my problem, but don't
know what to do about it.

/Flemming Jahn


Rick Jones skrev:

> > I have a problem with FTP and Jumbo frames, and would like to hear
> > if anybody else has experienced the same problem. The problem is
> > that when I enables jumbo frames support the ftp transfer rate
> > becomes very low compared with non-jumbo frames ftp
> > transferring. The actually numbers are :
>
> > Non-jumbo frames ftp transfer rate : 16.5 MB/s.
> > Jumbo frames ftp transfer rate : 30 kB/s.
>
> > When I do a simple drag-and-drop of a file between the 2 computers the
> > transfer time is the same regardless of enabling jumbo frames or not.
> > The problem isn't a switch problem, since I see the same thing using
> > a "crossed cable".
>
> > Is there anybody there have seen the same, or is there a configuration
> > I have forgot to setup ?.
>
> Some thoughs:
>
> *) does _everything_ between the two systems (inclusive) support
> JumboFrame?
>
> *) what is the TCP window size being used for the FTP data connection
> (you may need to do some packet tracing to find that)
>
> *) are there any TCP retransmissions (however one sees that in windows)
>
> *) IIRC there are times when Windows TCP will decide to do "direct
> I/O" (as it were) and hold-onto the user's buffer for DMA. Depending
> on the call the user made that may block the user until the data is
> ACKed by the remote, effectively turning the sliding window of TCP
> into stop-start.
>
> rick jones
>
> --
> No need to believe in either side, or any side. There is no cause.
> There's only yourself. The belief is in your own precision. - Jobert
> these opinions are mine, all mine; HP might not want them anyway... :)
> feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...


Posted by Rick Jones on July 17, 2006, 12:47 pm
If you were  Registered and logged in, you could reply and use other advanced thread options


> *) Yes, all the equipment supports jumbo frames.

> *) The TCP windows size is 64K.

> *) I am not quite sure about what you are saying about "user's buffer
> for DMA". Could you elaborate on that?

"Normally" when an application calls send, the kernel/stack makes a
copy of the user's buffer, and then schedules DMA from that copy. The
user is then free to keep running, doing whatever it wishes with his
buffer.

However, sometimes the MS TCP stack decides it does not want to copy.
Instead it will simply reference the user's buffer directly. Since
allowing the user to manipulat that buffer while the stack is sending
from it would lead to doubplusungoodness like data corruption, the
user is precluded from accessing the buffer. In the case of which I'm
thinking the user application is blocked from running entirely until
the data in that buffer has been ACKed by the remote. So, you lose
parallelism between the user and the stack/remote and things go
slowly.

> Another thing I have discovered is that I do not have any problem
> with TCP traffic when the MTU size is 4088 bytes, and I do not have
> any problems with UDP traffic ( MTU = 9014 bytes ).

While "Jumbo Frame" is not standardized, (I've seen all sorts of
examples of "Jumbo Frame" support asserted for what I wouldn't call
"real" Jumbo Frames), I thought the JF IP MTU was 9000 bytes, not
9014. I've heard of stacks calling 2048 bytes JF or 8192 but this
would be the first time I've seen 9014.

> I have tried to capture the traffic with ethereal, and there I can see
> that it takes 0.2 Sec to get an acknowledge from the receiving part
> when MTU=9014 bytes. Traffic with MTU=1518 bytes gives acknowledge
> within a few msec, so I expect that this is my problem, but don't
> know what to do about it.

0.2 seconds sounds like waiting for the standalone ACK timer. Are you
certain that the receiving TCP is advertising a 64KB window? And that
the sending TCP is sending multiple MSS segments at a time, keeping
that window full?

rick jones
--
Process shall set you free from the need for rational thought.
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Posted by fj on July 18, 2006, 7:34 am
If you were  Registered and logged in, you could reply and use other advanced thread options


Hi Rick

My NIC is an Intel Pro 1000, and the possible setting for Jumbo frames
is 9014 and 4088 bytes.

I have used Ethereal to determine the TCP window size, and it says that
the TCP windows size is 64K. I have even tried to increase the TCP
window size to 128K and that doesn't change anything.

That said, you did point me in the right direction, and I found
description of the 200 ms problem at Microsoft (
http://support.microsoft.com/kb/321098/ ). It turned out that it is
possible to disabled the delayed standalone ACK timer. When turned off,
the transfer rate increases to approx. 16 Mbytes/s.

Thanks for your help

Flemming Jahn




Rick Jones skrev:

> > *) Yes, all the equipment supports jumbo frames.
>
> > *) The TCP windows size is 64K.
>
> > *) I am not quite sure about what you are saying about "user's buffer
> > for DMA". Could you elaborate on that?
>
> "Normally" when an application calls send, the kernel/stack makes a
> copy of the user's buffer, and then schedules DMA from that copy. The
> user is then free to keep running, doing whatever it wishes with his
> buffer.
>
> However, sometimes the MS TCP stack decides it does not want to copy.
> Instead it will simply reference the user's buffer directly. Since
> allowing the user to manipulat that buffer while the stack is sending
> from it would lead to doubplusungoodness like data corruption, the
> user is precluded from accessing the buffer. In the case of which I'm
> thinking the user application is blocked from running entirely until
> the data in that buffer has been ACKed by the remote. So, you lose
> parallelism between the user and the stack/remote and things go
> slowly.
>
> > Another thing I have discovered is that I do not have any problem
> > with TCP traffic when the MTU size is 4088 bytes, and I do not have
> > any problems with UDP traffic ( MTU = 9014 bytes ).
>
> While "Jumbo Frame" is not standardized, (I've seen all sorts of
> examples of "Jumbo Frame" support asserted for what I wouldn't call
> "real" Jumbo Frames), I thought the JF IP MTU was 9000 bytes, not
> 9014. I've heard of stacks calling 2048 bytes JF or 8192 but this
> would be the first time I've seen 9014.
>
> > I have tried to capture the traffic with ethereal, and there I can see
> > that it takes 0.2 Sec to get an acknowledge from the receiving part
> > when MTU=9014 bytes. Traffic with MTU=1518 bytes gives acknowledge
> > within a few msec, so I expect that this is my problem, but don't
> > know what to do about it.
>
> 0.2 seconds sounds like waiting for the standalone ACK timer. Are you
> certain that the receiving TCP is advertising a 64KB window? And that
> the sending TCP is sending multiple MSS segments at a time, keeping
> that window full?
>
> rick jones
> --
> Process shall set you free from the need for rational thought.
> these opinions are mine, all mine; HP might not want them anyway... :)
> feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...


Similar ThreadsPosted
A GbE device not able to forward full-rate odd-byte frames November 2, 2005, 9:47 pm
Jumbo Ethernet Frames November 8, 2005, 3:44 pm
Jumbo frames practicalities? December 16, 2005, 1:56 pm
Another jumbo frames question March 20, 2007, 2:15 pm
PCMCIA GigE adapters with jumbo frames February 14, 2006, 2:46 pm
gigabit switch that supports jumbo frames? November 28, 2006, 8:46 pm
file transfer speed in a network October 11, 2006, 4:45 am
Optimize File Transfer on a 100Mbps LAN January 13, 2008, 3:09 am
VIA VT6102 Rhine-II + RealTek RTL8139C : file transfer hangs on Linux July 1, 2005, 3:36 pm
Easily add 4 Gb/s Ethernet link to FPGA systems for control & data transfer June 7, 2006, 11:54 am

other useful resources:
The Federal Communications Commission (FCC)
Telecommunications Industry Association
Electronic and Software Security Products and Services
International Telecommunication Union

Custom CGI Perl and PHP programming by 1-Script.com

Contact Us | Privacy Policy
The site map in XML format XML site map