Link: API Status Page for LDAS_MIT
915772875 STARTUP archiveLog /ldas_outgoing/logs/LDASmanager.log.html (file4) closed (archived as /ldas_outgoing/logs/archive/managerAPI/LDASmanager.915772874)
915772875 STARTUP execOverload overloaded exec
915772875 STARTUP openListenSock port 10002 (emergency) opened on gateway as sock7
915772875 STARTUP ::exec exec /usr/sbin/ifconfig -a
915772875 STARTUP mgr::perApiRunningStats read perAPIstats into ::mgr::apirunstats array
915772875 STARTUP getPid getting pids '27773 ' from lockfile /ldas_outgoing/diskcacheAPI/.diskcache.27773.lock
915772876 STARTUP mgr::bootstrapAPI diskcache pid is ''
915772876 STARTUP mgr::shutdownAPI obtained diskcache pid 27773 from lock file
915772876 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/diskcacheAPI/.diskcache.27773.lock
915772876 STARTUP mgr::postMortem stdout/stderr for previous run of diskcache API: http://18.120.0.97/ldas_outgoing/logs/archive/diskcacheAPI/diskcache.log.915772876
915772876 STARTUP exec exec uname -n
915772878 STARTUP mgr::bootstrapAPI bootstrapping diskcache API on dataserver
915772878 STARTUP mgr::bootstrapAPI Startup command for diskcacheAPI: 'alias cd=cd && cd /ldas_outgoing/diskcacheAPI && /bin/env RUNDIR=/ldas_outgoing HOST=dataserver PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/diskcacheAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /ldas/bin/diskcacheAPI >& /ldas_outgoing/diskcacheAPI/diskcache.log &'
915772888 STARTUP getPid getting pids '8445 ' from lockfile /ldas_outgoing/diskcacheAPI/.diskcache.8445.lock
915772888 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/diskcacheAPI/LDASdiskcache.911941305}
915772888 IDLE emergency_callback executed: set ::DISKCACHE_HASHFILE_NAME_BINARY .frame.cache
915772888 IDLE emergency_callback executed: set ::DISKCACHE_HASHFILE_NAME_ASCII frame_cache_dump
915772888 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo diskcache 1.10.1
915772889 STARTUP getPid pids 8445 obtained from ps for /ldas/bin/diskcache
915772889 STARTUP mgr::bootstrapSuccessEmail done. diskcache API is now running as pid: 8445
915772889 STARTUP getPid getting pids '27877 ' from lockfile /ldas_outgoing/frameAPI/.frame.27877.lock
915772890 STARTUP mgr::bootstrapAPI frame pid is ''
915772890 STARTUP mgr::shutdownAPI obtained frame pid 27877 from lock file
915772890 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/frameAPI/.frame.27877.lock
915772890 STARTUP mgr::postMortem stdout/stderr for previous run of frame API: http://18.120.0.97/ldas_outgoing/logs/archive/frameAPI/frame.log.915772890
915772890 STARTUP exec exec uname -n
915772892 STARTUP mgr::bootstrapAPI bootstrapping frame API on dataserver
915772892 STARTUP mgr::bootstrapAPI Startup command for frameAPI: 'alias cd=cd && cd /ldas_outgoing/frameAPI && /bin/env RUNDIR=/ldas_outgoing HOST=dataserver PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/frameAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /ldas/bin/frameAPI >& /ldas_outgoing/frameAPI/frame.log &'
915772932 STARTUP getPid getting pids '8549 ' from lockfile /ldas_outgoing/frameAPI/.frame.8549.lock
915772932 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/frameAPI/LDASframe.911941119}
915772932 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo frame 1.10.1
915772933 STARTUP getPid pids 8549 obtained from ps for /ldas/bin/frame
915772933 STARTUP mgr::bootstrapSuccessEmail done. frame API is now running as pid: 8549
915772933 STARTUP getPid getting pids '26093 ' from lockfile /ldas_outgoing/metadataAPI/.metadata.26093.lock
915772934 STARTUP mgr::bootstrapAPI metadata pid is ''
915772934 STARTUP mgr::shutdownAPI obtained metadata pid 26093 from lock file
915772934 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/metadataAPI/.metadata.26093.lock
915772934 STARTUP mgr::postMortem stdout/stderr for previous run of metadata API: http://18.120.0.97/ldas_outgoing/logs/archive/metadataAPI/metadata.log.915772934
915772934 STARTUP exec exec uname -n
915772934 STARTUP mgr::bootstrapAPI bootstrapping metadata API on metaserver
915772934 STARTUP mgr::bootstrapAPI Startup command for metadataAPI: 'alias cd=cd && cd /ldas_outgoing/metadataAPI && /bin/env RUNDIR=/ldas_outgoing HOST=metaserver PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/metadataAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /bin/env LD_PRELOAD=/ldas/lib/libpreload.so /ldas/bin/metadataAPI >& /ldas_outgoing/metadataAPI/metadata.log &'
915772939 STARTUP getPid getting pids '11100 ' from lockfile /ldas_outgoing/metadataAPI/.metadata.11100.lock
915772939 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/metadataAPI/LDASmetadata.911941119}
915772940 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo metadata 1.10.1
915772940 STARTUP getPid pids 11100 obtained from ps for /ldas/bin/metadata
915772940 STARTUP mgr::bootstrapSuccessEmail done. metadata API is now running as pid: 11100
915772940 STARTUP getPid getting pids '26191 ' from lockfile /ldas_outgoing/ligolwAPI/.ligolw.26191.lock
915772940 STARTUP mgr::bootstrapAPI ligolw pid is ''
915772940 STARTUP mgr::shutdownAPI obtained ligolw pid 26191 from lock file
915772940 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/ligolwAPI/.ligolw.26191.lock
915772940 STARTUP mgr::postMortem stdout/stderr for previous run of ligolw API: http://18.120.0.97/ldas_outgoing/logs/archive/ligolwAPI/ligolw.log.915772940
915772940 STARTUP exec exec uname -n
915772941 STARTUP mgr::bootstrapAPI bootstrapping ligolw API on metaserver
915772941 STARTUP mgr::bootstrapAPI Startup command for ligolwAPI: 'alias cd=cd && cd /ldas_outgoing/ligolwAPI && /bin/env RUNDIR=/ldas_outgoing HOST=metaserver PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/ligolwAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /ldas/bin/ligolwAPI >& /ldas_outgoing/ligolwAPI/ligolw.log &'
915772946 STARTUP getPid getting pids '11201 ' from lockfile /ldas_outgoing/ligolwAPI/.ligolw.11201.lock
915772946 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/ligolwAPI/LDASligolw.911941133}
915772946 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo ligolw 1.10.1
915772946 STARTUP getPid pids 11201 obtained from ps for /ldas/bin/ligolw
915772946 STARTUP mgr::bootstrapSuccessEmail done. ligolw API is now running as pid: 11201
915772946 STARTUP getPid getting pids '18270 ' from lockfile /ldas_outgoing/datacondAPI/.datacond.18270.lock
915772947 STARTUP mgr::bootstrapAPI datacond pid is ''
915772947 STARTUP mgr::shutdownAPI obtained datacond pid 18270 from lock file
915772947 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/datacondAPI/.datacond.18270.lock
915772947 STARTUP mgr::postMortem stdout/stderr for previous run of datacond API: http://18.120.0.97/ldas_outgoing/logs/archive/datacondAPI/datacond.log.915772947
915772947 STARTUP exec exec uname -n
915772947 STARTUP mgr::bootstrapAPI bootstrapping datacond API on datacon
915772947 STARTUP mgr::bootstrapAPI Startup command for datacondAPI: 'alias cd=cd && cd /ldas_outgoing/datacondAPI && /bin/env RUNDIR=/ldas_outgoing HOST=datacon PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/datacondAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /ldas/bin/datacondAPI >& /ldas_outgoing/datacondAPI/datacond.log &'
915772953 STARTUP getPid getting pids '29964 ' from lockfile /ldas_outgoing/datacondAPI/.datacond.29964.lock
915772953 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/datacondAPI/LDASdatacond.911941158}
915772953 IDLE emergency_callback bad key recieved: fe01d67a002dfa0f3ac084298142eccd
915772953 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo datacond 1.10.1
915772953 STARTUP getPid pids 29964 obtained from ps for /ldas/bin/datacond
915772953 STARTUP mgr::bootstrapSuccessEmail done. datacond API is now running as pid: 29964
915772953 STARTUP getPid getting pids '26291 ' from lockfile /ldas_outgoing/eventmonAPI/.eventmon.26291.lock
915772954 STARTUP mgr::bootstrapAPI eventmon pid is ''
915772954 STARTUP mgr::shutdownAPI obtained eventmon pid 26291 from lock file
915772954 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/eventmonAPI/.eventmon.26291.lock
915772954 STARTUP mgr::postMortem stdout/stderr for previous run of eventmon API: http://18.120.0.97/ldas_outgoing/logs/archive/eventmonAPI/eventmon.log.915772954
915772954 STARTUP exec exec uname -n
915772954 STARTUP mgr::bootstrapAPI bootstrapping eventmon API on metaserver
915772954 STARTUP mgr::bootstrapAPI Startup command for eventmonAPI: 'alias cd=cd && cd /ldas_outgoing/eventmonAPI && /bin/env RUNDIR=/ldas_outgoing HOST=metaserver PATH=/ldas/bin:/ldas/bin:/ldcg/bin:/bin:/usr/bin LD_LIBRARY_PATH=/ldas/lib/eventmonAPI:/ldcg/lib DB2INSTANCE=ldasdb GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 /ldas/bin/eventmonAPI >& /ldas_outgoing/eventmonAPI/eventmon.log &'
915772959 STARTUP getPid getting pids '11301 ' from lockfile /ldas_outgoing/eventmonAPI/.eventmon.11301.lock
915772959 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/eventmonAPI/LDASeventmon.911941167}
915772959 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo eventmon 1.10.1
915772960 STARTUP getPid pids 11301 obtained from ps for /ldas/bin/eventmon
915772960 STARTUP mgr::bootstrapSuccessEmail done. eventmon API is now running as pid: 11301
915772960 STARTUP getPid getting pids '15733 ' from lockfile /ldas_outgoing/cntlmonAPI/.cntlmon.15733.lock
915772960 STARTUP exec exec /bin/ps -fu ldas | grep cntlmon
915772960 STARTUP getPid child process exited abnormally
915772960 STARTUP mgr::bootstrapAPI cntlmon pid is ''
915772960 STARTUP mgr::shutdownAPI obtained cntlmon pid 15733 from lock file
915772960 STARTUP mgr::shutdownAPI deleted lock file: /ldas_outgoing/cntlmonAPI/.cntlmon.15733.lock
915772960 STARTUP mgr::postMortem stdout/stderr for previous run of cntlmon API: http://18.120.0.97/ldas_outgoing/logs/archive/cntlmonAPI/cntlmon.log.915772960
915772960 STARTUP exec exec uname -n
915772961 STARTUP mgr::bootstrapAPI bootstrapping cntlmon API on gateway
915772961 STARTUP ::exec exec /bin/env GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 RUNDIR=/ldas_outgoing LD_LIBRARY_PATH=/ldas/lib/cntlmonAPI:/ldcg/lib DB2INSTANCE=ldasdb cntlmonAPI >& /ldas_outgoing/cntlmonAPI/cntlmon.log &
915772961 STARTUP mgr::bootstrapAPI Startup command for cntlmonAPI: '/bin/env GLOBUS_GSI_CRED_DEBUG_LEVEL=0 GLOBUS_GSI_PROXY_DEBUG_LEVEL=0 GLOBUS_GSI_GSS_ASSIST_DEBUG_LEVEL=0 GLOBUS_ERROR_OUTPUT=1 GLOBUS_ERROR_VERBOSE=1 LD_PRELOAD= GLOBUS_GSI_OPENSSL_ERROR_DEBUG_LEVEL=0 GLOBUS_GSI_SYSCONFIG_DEBUG_LEVEL=0 GLOBUS_GSSAPI_DEBUG_LEVEL=0 RUNDIR=/ldas_outgoing LD_LIBRARY_PATH=/ldas/lib/cntlmonAPI:/ldcg/lib DB2INSTANCE=ldasdb cntlmonAPI >& /ldas_outgoing/cntlmonAPI/cntlmon.log &'
915772966 STARTUP getPid getting pids '5779 ' from lockfile /ldas_outgoing/cntlmonAPI/.cntlmon.5779.lock
915772966 STARTUP exec exec /bin/ps -fu ldas | grep cntlmon
915772966 STARTUP mgr::bootstrapSuccessEmail done. cntlmon API is now running as pid:
915772966 STARTUP bgLoop Looping process cntlmonping started
915772966 STARTUP bgLoop Looping process datacondping started
915772966 STARTUP bgLoop Looping process diskcacheping started
915772966 STARTUP bgLoop Looping process eventmonping started
915772966 STARTUP bgLoop Looping process frameping started
915772967 STARTUP bgLoop Looping process ligolwping started
915772967 STARTUP bgLoop Looping process metadataping started
915772967 STARTUP leakLogger inital size of manager API: 31688 kB
915772967 STARTUP bgLoop Looping process etchosts started
915772967 STARTUP bgLoop Looping process memflag started
915772967 STARTUP bgLoop Looping process statpagefile started
915772967 STARTUP bgLoop Looping process statpage started
915772968 STARTUP bgLoop Looping process hidedirs started
915772968 STARTUP bgLoop Looping process leapsecs started
915772968 STARTUP bgLoop Looping process bslock started
915772968 STARTUP bgLoop Looping process blockage started
915772968 STARTUP bgLoop Looping process createassistant started
915772968 STARTUP bgLoop Looping process writestatus started
915772968 STARTUP bgLoop Looping process expireusers started
915772968 STARTUP mgr::removeUser user 'lazz' removed from 'users.queue'
915772968 STARTUP mgr::findExpiredUsers Subject: LDAS-MIT managerAPI user account 'lazz' about to expire!; Body: ldas-mit user account: 'lazz' expires at Mon Dec 08 00:00:00 EST 2008. account belongs to 'Albert Lazzarini'. Last known email address was: 'lazz@ligo.caltech.edu' user has been notified via email of account status.
915772968 STARTUP mgr::removeUser user 'reedtracie' removed from 'users.queue'
915772968 STARTUP mgr::findExpiredUsers Subject: LDAS-MIT managerAPI user account 'reedtracie' about to expire!; Body: ldas-mit user account: 'reedtracie' expires at Sat Jan 10 00:00:00 EST 2009. account belongs to 'Tracie M. Reed'. Last known email address was: 'tmr016@latech.edu' user has been notified via email of account status.
915772969 STARTUP mgr::removeUser user 'gonzalez' removed from 'users.queue'
915772969 STARTUP mgr::findExpiredUsers Subject: LDAS-MIT managerAPI user account 'gonzalez' about to expire!; Body: ldas-mit user account: 'gonzalez' expires at Sun Jan 11 00:00:00 EST 2009. account belongs to 'Gabriela Gonzalez'. Last known email address was: 'gonzalez@lsu.edu' user has been notified via email of account status.
915772969 STARTUP mgr::removeUser user 'shantanu' removed from 'users.queue'
915772969 STARTUP mgr::findExpiredUsers Subject: LDAS-MIT managerAPI user account 'shantanu' about to expire!; Body: ldas-mit user account: 'shantanu' expires at Thu Jan 08 00:00:00 EST 2009. account belongs to 'Shantanu Desai'. Last known email address was: 'desai@gravity.psu.edu' user has been notified via email of account status.
915772969 STARTUP bgLoop Looping process diskusagelimit started
915772969 STARTUP exec exec df -m /export/ldcg_server/ldas_outgoing/managerAPI
915772969 STARTUP bgLoop Looping process logrotate started
915772969 STARTUP bgLoop Looping process livejobpush started
915772970 IDLE exec exec /ldcg/bin/lsof -b -p 5544
915772971 IDLE openListenSock port 10001 (operator) opened on gateway as sock5
915772971 IDLE setResourceLimit vmemoryuse=unlimited; datasize=unlimited; core=unlimited; getRLimit on maxproc error:'Invalid or unsupported resource name: maxproc'; descriptors=65536; getRLimit on memorylocked error:'Invalid or unsupported resource name: memorylocked'; filesize=unlimited; cputime=unlimited
915772971 IDLE mgrGlobusChannel::init cmd gt_xio_socket -server mgrGlobusChannel::accept -gridmap_file -myaddr gateway 10031
915772972 IDLE mgrGlobusChannel::init Globus channel gtxiosock0 opened for gsi service at 10031
915772972 IDLE mgrGlobusChannel::init cmd gt_xio_socket -server mgrGlobusChannel::accept -gridmap_file -myaddr gateway 10030
915772972 IDLE mgrGlobusChannel::init Globus channel gtxiosock1 opened for gsi service at 10030
915772972 IDLE emergency_callback executed: eval {archiveIndex /ldas_outgoing/logs/archive/cntlmonAPI/LDAScntlmon.911941322}
915772972 IDLE emergency_callback executed: mgr::pushFTPandHTTPinfo cntlmon 1.10.1
915773075 IDLE mgr::addUser new user added to users.queue: '-name jgiaime -password ***** -email jgiaime@ligo.phys.lsu.edu -fullname {Joseph A. Giaime} -phone 1-225-578-5794 -expires 2010-1-14 {}'
915773075 IDLE mgr::addUser new user added to ::QUEUE(USERS): '-name jgiaime -password ***** -email jgiaime@ligo.phys.lsu.edu -fullname {Joseph A. Giaime} -phone 1-225-578-5794 -expires 2010-1-14 {}'
915773075 IDLE emergency_callback user info updated successfully