And so why are FTP uploads soo deadening? I am using filezila as a client.

I have like x mb in 1000 files and I can upload each individual file with 300-500kb/s yet the whole upload is incredibily tiresome due to the queueing procedure that occurs equally files are uploaded. For every singe file the customer performs all kind of commands and connection operations before actually uploading.

Is at that place no way to skip over these commands? I am new to ftp clients/uploads/websites etc Is this standard do? Is this just the mode ftp uploads work? Don't you get bored waiting like 20 minutes for 8-10 mb?

How can I efficiently upload 100 mb or more?

asked Jun xiii, 2011 at 17:41

3

  • Transferring a large number of pocket-size files is much less efficient for any protocol, not merely FTP. It's simply much more than noticeable over the internet (or a WAN) because the latency makes the command overhead swamp the actual transfer rate.

    Jun fourteen, 2011 at 17:59

  • @afrazier theoretically non. Information technology depends on the nature of the protocol in use, which doesn't support pipelining. For example, pipelined http was designed to reduce latency

    Oct 21, 2014 at 20:05

  • @djechelon: Pipelining helps, just you nevertheless have the problem of control overhead and about pipeline implementations limit the number of active simultaneous requests (because overhead tin still swamp the payload eventually).

    Oct 21, 2014 at twenty:14

iv Answers 4

Sadly, this is the way that FTP functions. To efficiently transfer lots of modest files, either annal them locally, transmit the unabridged annal via ftp, and so unarchive the files on the remote auto, or turn on simultaneous uploads, where the client is uploading ten files at once. This will help to fully saturate your upload link.

answered Jun 13, 2011 at 17:45

user avatar

two

  • with an avg. of 100kb per file they are non pocket-sized files.

    Jun thirteen, 2011 at 17:53

  • @Captain Giraffe: On almost WAN/Net links, 100 kB is nonetheless small enough that latency swamps throughput when it comes to uploading.

    Jun 14, 2011 at 17:56

Take yous tried compressing the files locally so uncompressing them on the server? So you lot'd simply have to transfer one small(er) file.

If it's applicative, y'all could besides only copy files that have changed since your previous upload. Tools similar rsync (if you accept ssh admission) and robocopy (if information technology's a windows server) could assist you practise this.

answered Jun 13, 2011 at 17:43

user avatar

I know this is an onetime mail service, and there isn't a lot yous tin do at the application layer since FTP relies on TCP for sending the bytes. There are few things though

  1. Enable simultaneous uploads as mentioned above (this helps employ more of your bandwidth since it takes fourth dimension for TCP to "ramp up" each connection, so doing it in parallel is more efficient)
  2. Archive the file as mentioned to a higher place (again, considering it allows TCP to ramp up over time instead of individually for each file)
  3. Check out SuperTCP: it's a new production that (total disclosure) I'1000 helping build that will optimize TCP upload, which tin can help FTP go faster in a lot of cases. We're launching a Beta soon and would love to have you help us examination it!

answered Oct 8, 2014 at 17:53

user avatar

I use Car FTP Manager. Information technology runs multiple simultaneous FTP transfers so the entire bandwidth link can be used:

Auto FTP Manager makes it easy to schedule and automate your FTP transfers. Use Auto FTP Manager to connect to any FTP server and automatically upload and download files. Programme and automate your workflow. Permit your PC move or synchronize files betwixt PC to FTP Server, PC to PC, and FTP Server to FTP Server, automatically according to a schedule...

... Car FTP Manager is multi-threaded, allowing yous to open connections to multiple FTP servers at the same time. The programme can transfer files in the groundwork while you work on other tasks.

user avatar

Diogo

29.5k 64 aureate badges 146 silver badges 220 bronze badges

answered Jun 14, 2011 at 16:35

user avatar