ColdFusion 10.0  -  Bug 3318104

Created on Sunday, August 26, 2012

Login for more options

Title

Service Temporarily Unavailable after arbitrary amount of time + heavy usage

Description

Problem Description:

The problem is described in detail in this thread:

http://forums.adobe.com/message/4649986#4649986

CF 10 under Windows 2008 x64 and IIS7 will periodically start serving up the following messages:

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Jakarta/ISAPI/isapi_redirector/1.2.32 ()

Restarting the web publishing service causes the problem to go away.

Steps to Reproduce: Cannot reproduce on demand. Happens every 1-4 weeks on a server with 14 or so CF applications and several thousand active users.

Actual Result:

Expected Result:

Any Workarounds:

Test Configuration

My Hardware and Environment details: Win 2008 Server x64 (Web or Enterprise edition), 8 GB RAM, CF10 Standard

App Language(s) English
OS Language(s) English
Platform(s) Win 2008 Server R2 64 bit
Browser(s) Browsers All

Notes (34)

  • Rupesh Kumar

    11:08:43 AM GMT+00:00 Nov 12, 2013

    ************ Please read this *******************
    If configuring your server as per http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration does not solve your problem, could you please get in touch with us at "cf.install AT adobe DOT com" and we would work with you to resolve this issue.

  • xstrava

    7:47:02 AM GMT+00:00 Oct 8, 2013

    We are running the same exact configuration as noted above except we have 16 GB of RAM on a virtual server. We upgraded to ColdFusion 10 because our our last web server (ColdFusion 9) was compromised. I installed all Windows updates and have CF 10 up to Update 11. The problem is frustrating because the only way I know how to resolve it it to restart the IIS Service. Hopefully this will be addressed soon as I am losing too much sleep over this issue.

  • williakc

    11:39:33 AM GMT+00:00 Oct 1, 2013

    Tried tuning like in the blog Windows server 2008, IIS 7.5 Coldfusion 10 patched to patch level 11
    http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration
    did not resolve our problem

  • Rupesh Kumar

    10:47:16 PM GMT+00:00 Aug 13, 2013

    **** Please check the comment below by Vishal Jaswal *****
    You need to tune the server to address this. We have worked with quite a few customers and confirmed that tuning the server as per the following blog entry actually resolves the problem.
    http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

    The same blog entry should be applicable for Apache as well.
    Does this not resolve your problem?

  • seedingideas

    7:24:06 PM GMT+00:00 Aug 13, 2013

    We have this problem too. We are going to go out of business if we do not get this resolved!

  • hsuanita88

    12:14:42 PM GMT+00:00 Aug 12, 2013

    We have been having this problem too. Please fix this issue. Thanks.

  • hsuanita88

    12:14:07 PM GMT+00:00 Aug 12, 2013

    We have been having this problem too. Please fix this issue. Thanks.

  • Vishal Jaswal

    6:42:56 AM GMT+00:00 Jul 17, 2013

    Made Changes in the following and the server is stable.

    worker.properties

    worker.cfusion.connection_pool_size = 500
    worker.cfusion.connection_pool_timeout = 60

    server.xml

    maxThreads=500 and connectionTimeout="60000"

    Followed blog : http://blogs.coldfusion.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration

  • BigJ57

    7:01:24 PM GMT+00:00 Jul 12, 2013

    Agree that Adobe should fix this but did anyone try the BonCode connector instead of the ISAPI to see whether that resolves the problem?

  • Solodev_1

    6:03:40 AM GMT+00:00 Jul 12, 2013

    Please fix this bug! What is the status?

  • jbliss0J6c

    10:30:38 AM GMT+00:00 Jul 9, 2013

    What is the status of this bug? It appears to still exist as of Update 11.

  • Asha K S

    8:11:49 AM GMT+00:00 Jun 13, 2013

    HI ParaEl
    Can you please email your contact mail id to asha@adobe.com so that we can contact you to resolve this issue.

    Thanks.

  • ParaEl

    7:17:24 AM GMT+00:00 May 29, 2013

    Using the method described by Asha K S did not work for us. A more detailed explanation of our problem can be found here: https://bugbase.adobe.com/index.cfm?event=bug&id=3530880
    It may be related to the problems mentioned here by others as well.

  • Asha K S

    4:08:21 AM GMT+00:00 May 27, 2013

    Can you please try changing your settings as suggested in http://www.webtrenches.com/post.cfm/resolve-stability-problems-and-speed-up-coldfusion-10

  • Neo101

    8:59:13 PM GMT+00:00 Apr 29, 2013

    Thanks.
    Server details:
    VM Ware cloud.
    Windows server 2008 R2 web incl. all updates. IIS – one active site.
    Xeon CPU 2650 @1.00Ghz (2 processors), 6 GB Ram.
    Coldfusion 10 Standard - 10,0,9,284568 all latest updates.
    Max simultaneous template requests set to 10 (changed a few times – no affect). Request timeout: 60
    Server load – very low, 30 users, 0-50% CPU, 1.3 – 2 GB RAM.
    Web server connector buffer was disabled (because using cfflush) in an attempt to solve problem – no change.
    2 weeks ago completely reinstalled IIS role + CF10 + updates, still throwing the error.
    Any idea what to do next?

  • Asha K S

    2:34:00 AM GMT+00:00 Apr 26, 2013

    Can you please let us know a bit more information about your setup -
    like connector pool configuration along with Timeout values,OS resource usage, handle usage, CF admin details etc so that we can try and repro the issue on our end .Thanks.

  • Neo101

    9:29:47 AM GMT+00:00 Apr 25, 2013

    It happens to me twice a week, completely unpredictable.
    Does anyone know if this bug also affects CF10 running on Apache? I'm considering disabling IIS and setting Apache instead. Thx.

  • stevenj@sca-corp.com

    6:16:14 AM GMT+00:00 Apr 23, 2013

    This was happening to us apparently with large number of website hits, but not anymore. NOW it happens when PCI scans our servers for compliance. It was happening every month, now about every other month. It just happened this morning on 4/23/13, so the bug hasn't been entirely fixed.

  • etnevels1

    8:52:52 AM GMT+00:00 Feb 19, 2013

    Exact server here. Only have this with CF10 - have installed all updates. What I notice is this only happens on sites that have several bad page requests. For example if the client is new to us and has some links out there to non-existent pages, then that site will crash several times a day after a few of these bad GET page requests are made. Here is a recent log entry that killed the site: 2013-02-19 16:06:41 W3SVC26 FULLPARTNER5 76.12.79.46 GET /wp-login.php - 91.207.7.238 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US;+rv:1.9.1.3)+Gecko/20090824+Firefox/3.5.3+GTB5 - - 200 0 235 93
    Notice the page is /wp-login.php,obviously a page from the previous site that does not exist on the new site.
    We have only used CF10 for about 3 weeks. No such problem in CF9 or CF8 on these exact same sites. Clearly a CF10 issue, clearly a jakarta issue. I shudder to think what will happen when the PCI guys hit a site for a vulnerability scan - the always try to hit a hundred bad pages to see what the server does. Probably the server will crash! So far we have not had a PCI scan on this server....

  • guillermo_dewey

    6:56:29 AM GMT+00:00 Dec 3, 2012

    having the exact same problem.. cf9 and cf10.. with cf10 happens less often.. problem solves afer restaring the cf service

    The service is unavailable.
    Service Temporary Unavailable!
    The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
    Jakarta/ISAPI/isapi_redirector/1.2.32 ()

    in the logs I am getting

    Dec 2, 2012 3:33:04 AM org.apache.catalina.core.StandardWrapperValve invoke
    SEVERE: Servlet.service() for servlet [CfmServlet] in context with path [/] threw exception [ROOT CAUSE:
    java.lang.OutOfMemoryError: Java heap space
    ] with root cause
    javax.servlet.ServletException: ROOT CAUSE:
    java.lang.OutOfMemoryError: Java heap space

    for that I am increasing the setting

    Maximum JVM Heap Size WITH OUT ANY solution

  • gonsalas

    5:56:31 AM GMT+00:00 Nov 15, 2012

    I have the same issue please help me to solve this because i need the server online for tomorrow and i have 2 weeks looking for an answer.... Thanks

  • Hemant Khandelwal

    10:31:38 AM GMT+00:00 Oct 31, 2012

    We are most likely releasing the fix by end of this week. We needed time to do additional testing plus involve community via private release to test the fix. Thanks for your support and feedback.

  • *** Tyson ***

    7:06:07 AM GMT+00:00 Oct 31, 2012

    Same exact issue for me with one of my biggest clients. Constant crashes on our ColdFusion 10 server multiple times per day. Event viewer is riddled with constant errors (thousands per day) from the w3wp.exe service and the ISAPI connector. I was able to seemingly relieve the situation somewhat by disabling IIS 7.5's "Rapid-Fail Protection" on the application pools (which I can't imagine is advisable for a production environment), but that only seems to have decreased the frequency of the crashes slightly - from a dozen times each day, down to only a few times per day. Completely unacceptable.

    It absolutely astounds me that Adobe has not halted in their tracks and prioritized every single reputatble development resource with developing and releasing an effective resolution to this problem. Honestly, it gives me cause (for the first time in 17 years of ColdFusion development and support) to consider migrating away from ColdFusion to a different platform. Updater 3 was apparently a disaster (thank God I checked for community feedback before installing it), but I've not heard a single follow up about the fix to this problem since the Updater 3 was recalled by Adobe - and that was a couple weeks ago.

    How on earth can an absolutely critical bug like this remain open and unaddressed for so long without any resolution!? Adobe, you've -completely- dropped the ball on this and disappointed droves upon droves of loyal customers, developers, and community members. Our clients are angry. Not only are they losing money and credibility every single day that this bug remains unfixed, but their confidence and support of ColdFusion as a platform is dwindling. What in the world is holding up the completion and release of a prioritized hot fix for this issue!? Forget about bundling it up with a bunch of other fixes at this point. This bug is critical enough to stand on its own and be addressed immediately!

    Unbelievable.

  • *** Tyson ***

    6:32:02 AM GMT+00:00 Oct 31, 2012

    I don't understand -WHY- this issue hasn't been fixed yet. It's bringing down my Fortune 200 client's server multiple times per day. Can Adobe AT LEAST provide an ETA on a fix for these connector issues!?

  • Vamseekrishna Manneboina

    9:04:05 PM GMT+00:00 Oct 8, 2012

    Thi fix for this issue will be available in the next update which we plan to release very soon.

  • Ged_mc

    8:36:47 AM GMT+00:00 Oct 6, 2012

    This is a very common platform
    Adobe must fix this ASAP or offer a refund - since CF10 is useless if this unstable

  • kenreiss

    2:32:12 PM GMT+00:00 Oct 1, 2012

    I am having the same thing with CF10 Update 2. Installed it yesterday and failing already today!

    The error messages I see are identical to those shown in this bug fix. Can't discern exactly what users were doing at the time, but only resolution was to restart CF service.

    PLEASE fix this, as it is causing real issues on our production server. Our customers are VERY angry.

    Perhaps these links will help:
    http://forums.adobe.com/message/4465125
    http://forums.adobe.com/message/4619002#4619002

  • roland.collins

    9:27:47 AM GMT+00:00 Sep 18, 2012

    Also receiving this error. Look at the number of votes on this issue! CF10 has been an utter disaster for us so far.

  • John Cambridge

    9:08:00 AM GMT+00:00 Sep 18, 2012

    Also, I would like to say, this happens several times a week for us.

  • John Cambridge

    9:06:50 AM GMT+00:00 Sep 18, 2012

    Still having this problem in CF10 Update 2

  • Kiran Sakhare

    4:34:51 AM GMT+00:00 Aug 31, 2012

    We also need some information on server setup.
    1. How many sites in the IIS are configured with CF
    2. Have you tried giving the timeout value in wsconfig->advanced option.
    3. Enable the cachewebserverpath option in the server admin settings page and verify if you can repro the issue.
    4. Give us server load information about number of request at normal and peak time.

  • Evelin Varghese

    10:36:31 PM GMT+00:00 Aug 29, 2012

    We are trying to reproduce this issue. Any Sample code that reproduces the issue will be really helpful.
    We can simulate continuous http request firing to replicate the issue.

  • AlHolden

    11:43:40 AM GMT+00:00 Aug 29, 2012

    coldfusion-error.log fills up with ~5mb of these entries on each occurance:
    Aug 29, 2012 11:24:58 AM org.apache.coyote.ajp.AjpProcessor getRealPathFromServer
    SEVERE: Error in getRealPathFromConn
    java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:124)
    at org.apache.coyote.ajp.AjpProcessor.getRealPathFromServer(AjpProcessor.java:391)
    at org.apache.coyote.ajp.AbstractAjpProcessor.getRealPath(AbstractAjpProcessor.java:1112)
    at org.apache.naming.resources.FileDirContext.doGetRealPath(FileDirContext.java:198)
    at org.apache.naming.resources.BaseDirContext.getRealPath(BaseDirContext.java:412)

  • @bittersweetryan

    11:23:04 AM GMT+00:00 Aug 29, 2012

    This happens to me when I have a http request with a large paylod in the content, for instance doing a HTML5 XMLHTTPRequest with a file in the request.

Duplicate ID
Reported By Aquitaine

Status

State Closed
Status Fixed
Reason

Importance

Priority 4-Very High
Frequency Some users will encounter
Failure Type Crash
Product Area Web Container (Tomcat)

Build

Found In Build Final
Fixed In Build Latest

Attachments (1)

Votes (39)

  • AustinValley

    6:44:53 AM GMT+00:00 Nov 18, 2013

    I just got this for the first time today. About 1500 hours of cpu time without reboot/restart. Our configuration is CF10 with Hotfix level 10. This effected our scheduled jobs which are critical. CF10 64-bit on Win2008with 24 gb ram. The reputation of ColdFusion is on the line in this 6+ enterprise server licensed organization.

  • DCwebGuy

    10:40:57 AM GMT+00:00 Nov 13, 2013

    I am having the exact same problems despite performing all the suggested connector tuning. I have the latest CF10 Update 11 running, clean install, medium sized site, and yet continue to see 503 error messages a few times a week. Windows 2008 R2 (VMware), IIS 7.5. Never, ever had these problems with CF9 on 32-bit Windows 2003/IIS6. Thinking about falling back to CF9 but would really prefer for CF10 to work given all the time and $$$ spent. Will try to get in touch with Adobe privately, something I've never had to do in 16 years of working with CF. Bummer.

  • cannd

    7:51:15 AM GMT+00:00 Sep 30, 2013

    This is happening with us also. We just upgraded to cf10 a couple of months ago. It is costing us $$$$. I have followed the advice for tuning the connector but problem continues.

  • Digitalhappy

    3:40:38 AM GMT+00:00 Aug 20, 2013

    I'm seeing this about 3 or four times a week on Windows 2008 server running the latest patched version of CF10 (update 11 I believe).
    It's driving me crazy and proving to be a very frustrating time as I try and work out what is actually going on.

  • seedingideas

    7:25:05 PM GMT+00:00 Aug 13, 2013

    Our server goes down multiple times per day - with unreliable servers, we are losing clients on a daily basis. We are going to go out of business!

  • sdsinc_pmascari

    5:00:35 PM GMT+00:00 Jul 20, 2013

    We get this all the time. Some times if we leave the server long enough it will come back on its own. Other times, the site with the heaviest usage will stay running but all other sites will go down. Weird.

  • trent_sc

    5:33:42 AM GMT+00:00 Jul 18, 2013

    We have also experienced this random outage with the same isapi redirect error.

  • henrylearn2rock

    3:02:27 PM GMT+00:00 Jul 12, 2013

    We just launched a few CF10 production servers. We hope we won't run into this. This is marked Fixed, but when will this fix be available?

  • Solodev_1

    6:02:59 AM GMT+00:00 Jul 12, 2013

    Please fix this bug. It is costing us much money.

  • jbliss0J6c

    10:24:46 AM GMT+00:00 Jun 12, 2013

    We were down for thirty minutes today because of this. Lost lots of money.

  • ParaEl

    8:00:47 AM GMT+00:00 May 28, 2013

    We are hitting this problem as well.

  • sjibben

    11:24:30 PM GMT+00:00 May 10, 2013

    Running into this problem with CF 10u9 on Windows Server 2008 R2 64-bit.
    I had to run this to get it working (DOS Prompt elevated to Administrator security level):
    {cfusion-install}\cfusion\bin\connectors\Upgrade_all_connectors.bat

  • Rafael Salomon

    10:07:14 AM GMT+00:00 Mar 20, 2013

    had same problem, reverted to cf9. please fix asap.

  • Tamakazura

    9:24:56 AM GMT+00:00 Feb 25, 2013

    Regret the upgrade to version 10 - please get a fix for this ASAP!!

  • vadnick

    7:15:07 AM GMT+00:00 Jan 11, 2013

    Still experiencing this issue. Service crashes 5-10 times per day, primarily during business hours when traffic and usage is higher.

  • jdbrady67

    4:07:28 PM GMT+00:00 Nov 1, 2012

    If this isn't released by tomorrow, I will need someone from Adobe to send me the fix that is ebing used by testers. I have numerous CF 10 servers that I'm on the verge of migrating to and have one if Production that has this issue that is highly visible and need immediate resolve.

  • *** Tyson ***

    6:41:08 AM GMT+00:00 Oct 31, 2012

    Fix this bug immediately! It is absolutely KILLING any credibility in ColdFusion 10 as a stable enterprise platform. My Fortune 200 client's server is crashing multiple times per day as a result of this issue.

  • azlonghorn

    7:37:35 AM GMT+00:00 Oct 30, 2012

    seeing this after migrating prod servers to cf10

  • CFdevFromJava

    3:38:52 PM GMT+00:00 Oct 29, 2012

    I have been working on this since July. I purchase 3 CF10 licenses (1)enterprise and (2)standards and invested in a new server, all to no avail. I have already spent $1500 in support and am receiving heat about the cost and have had to revert back to CF9. This should be a high priority for Adobe! Updates 1 & 2 fixes some others issues with standard but when will this update be available?

  • A.B.

    10:55:19 AM GMT+00:00 Oct 26, 2012

    We're unable to use web service and it brings the website down.

  • nature_girl151

    11:37:26 AM GMT+00:00 Oct 8, 2012

    Having the same issues here - I have update 2 installed and get this error 3 or 4 times per day.
    Th server is a dev server and barely used. We cannot move to production with this error occurring

  • loftx

    4:08:44 AM GMT+00:00 Oct 5, 2012

    I believe we are experiencing this issue as well which was causing a production server to crash with the symptoms reported by others every hour or so at times.

  • kenreiss

    2:31:28 PM GMT+00:00 Oct 1, 2012

    I am having the same thing with CF10 Update 2. Installed it yesterday and failing already today!

    The error messages I see are identical to those shown in this bug fix. Can't discern exactly what users were doing at the time, but only resolution was to restart CF service.

    PLEASE fix this, as it is causing real issues on our production server. Our customers are VERY angry.

    Perhaps these links will help:
    http://forums.adobe.com/message/4465125
    http://forums.adobe.com/message/4619002#4619002

  • gammajring

    5:57:50 AM GMT+00:00 Sep 20, 2012

    This is happening on our CF10 production server as well.

  • roland.collins

    9:28:11 AM GMT+00:00 Sep 18, 2012

    Also experiencing the issue. Where is Adobe's commentary on this?

  • John Cambridge

    9:06:18 AM GMT+00:00 Sep 18, 2012

    We have this problem also. This is terrible. CF10 has been so buggy. We've had to make a program that continuously checks if the server's up and restarts the CF10 Application Service.

  • Joshua Cyr

    9:34:10 AM GMT+00:00 Sep 14, 2012

    My low traffic server has this issue as well. Maybe once a month or more? I have no known trigger as of yet.

  • TMG2004

    2:18:39 PM GMT+00:00 Sep 7, 2012

    After upgrading form CF 9.1 Standard to CF 10 we have seen the same issues.

  • Carl Von Stetten

    1:56:58 PM GMT+00:00 Sep 7, 2012

    I, like Adam and Miguel, am not personally having this issue (yet) because I am still in the process of configuring our new CF10 server (Win2K8 R2) and don't have any users hitting it in production yet, But I'm very concerned that I may encounter the same issues.

  • Kory Gorsky

    8:08:00 AM GMT+00:00 Sep 5, 2012

    Happening on our Production Server.

  • newaite

    6:48:28 AM GMT+00:00 Sep 4, 2012

    Had this and other problems.

    I had to revert to CF9

  • Dmitry Yakhnov

    6:29:18 PM GMT+00:00 Aug 30, 2012

    Same problem at Hostek with CF10.

  • Miguel-F

    1:38:43 PM GMT+00:00 Aug 29, 2012

    I, like Adam, am not personally having this issue because I have not upgraded to CF10 yet (still running CF9) but I am hearing lots of other people with similar issues. For what it's worth here is another forum discussion regarding this: http://forums.adobe.com/message/4657976#4657976

  • Adam Cameron.

    1:24:35 PM GMT+00:00 Aug 29, 2012

    I'm not having this problem (I don't run Win2k8), but I've heard enough of it happening that it seriously needs looking at.

    --
    Adam

  • Peter 456

    12:38:33 PM GMT+00:00 Aug 29, 2012

    We are seeing server hangs on CF10 that may be related to this.

  • AlHolden

    11:45:15 AM GMT+00:00 Aug 29, 2012

    breaks when calling flash forms which perform data loads from components on init

  • lg84

    11:25:45 AM GMT+00:00 Aug 29, 2012

    We were live on Coldfusion 10 Standard Edition on 2 Windows Server 2008 VM (32 bit) for almost a week. Last night, both servers stopped responding at the same time. When a server was restarted, it would run ok initially on restart but then would lock up within the next 10 minutes. IIS logs show increased traffic from bots during that time. We had to revert back to Coldfusion 9.

  • @bittersweetryan

    11:19:44 AM GMT+00:00 Aug 29, 2012

    This is happening to me as well, under a small testing load. It's holding up a large state project from going into production.

  • klevenson

    10:59:52 AM GMT+00:00 Aug 29, 2012

    We have 5 server with CF10 Enterprise that exhibit this same behavior. Within 2 hours of heaver traffic, a buffer overflow error is registered in the isapi_redirect.dll

Your session has expired! Click to login
Current form data will be preserved

Cancel