GlobalSearch::QueueManager Class Reference

The QueueManager monitors the running jobs and updates Structure status. More...

#include <globalsearch/queuemanager.h>

Public Slots
void	reset ()
void	killStructure (Structure *s)
void	appendToJobStartTracker (Structure *s)
void	addManualStructureRequest (int requests=1)
QList< Structure * >	getAllPreoptimizingStructures ()
QList< Structure * >	getAllRunningStructures ()
QList< Structure * >	getAllOptimizedStructures ()
QList< Structure * >	getAllDuplicateStructures ()
QList< Structure * >	getAllStructures ()
QList< Structure * >	lockForNaming ()
void	unlockForNaming (Structure *s=0)
Signals
void	movedToQMThread ()
void	structureStarted (GlobalSearch::Structure *s)
void	structureSubmitted (GlobalSearch::Structure *s)
void	structureKilled (GlobalSearch::Structure *s)
void	structureUpdated (GlobalSearch::Structure *s)
void	structureFinished (GlobalSearch::Structure *s)
void	needNewStructure ()
void	newStatusOverview (int optimized, int running, int failing)
Public Member Functions
	QueueManager (QThread thread, OptBase parent)
virtual	~QueueManager ()
Protected Slots
void	checkLoop ()
void	senderHasFinishedPreoptimization ()
void	addStructureToSubmissionQueue (GlobalSearch::Structure *s, int optStep)
void	addStructureToSubmissionQueue (GlobalSearch::Structure *s)
void	moveToQMThread ()
void	setupConnections ()
Protected Member Functions
void	updateQueue ()
void	updateStructure (Structure *s)
void	startPreoptimization ()
void	startJob ()
void	stopJob (Structure *s)
void	checkPopulation ()
void	checkRunning ()
void	handleOptimizedStructure (Structure *s)
void	handleStepOptimizedStructure (Structure *s)
void	handleWaitingForOptimizationStructure (Structure *s)
void	handleInProcessStructure (Structure *s)
void	handleEmptyStructure (Structure *s)
void	handlePreoptimizingStructure (Structure *s)
void	handleUpdatingStructure (Structure *s)
void	handleErrorStructure (Structure *s)
void	handleSubmittedStructure (Structure *s)
void	handleKilledStructure (Structure *s)
void	handleRemovedStructure (Structure *s)
void	handleDuplicateStructure (Structure *s)
void	handleRestartStructure (Structure *s)
Protected Attributes
OptBase *	m_opt
	Cached pointer to main optbase class.
QThread *	m_thread
	Pointer to the thread where the queuemanager lives.
Tracker *	m_tracker
	Convenience pointer to m_opt->tracker().
Tracker	m_needPreOptTracker
Tracker	m_preOptTracker
	Tracks which structures are currently preoptimizing.
Tracker	m_runningTracker
	Tracks which structures are currently running.
Tracker	m_jobStartTracker
	Tracks which structures are queued to be submitted.
Tracker	m_newStructureTracker
int	m_requestedStructures
	Number of structure requests pending.
bool	m_isDestroying
	Boolean set to true while the destructor is running.
QDateTime *	m_lastSubmissionTimeStamp
	Used to throttle job submissions.

Detailed Description

The QueueManager monitors the running jobs and updates Structure status.

Author:: David C. Lonie

The QueueManager creates a local queue and monitoring system control the submission of Structures to an optimization engine or queue.

For basic usage, connect the needNewStructure() signal to slot that will generate a new Structure and submit it the following way:

// lockForNaming() returns a list of all structures that the main
// tracker is aware of. It also locks a naming mutex to prevent
// simultaneous naming of Structures, avoiding duplicate
// Structure indices, ID numbers, etc.
QList<Structure*> allStructures = m_queue->lockForNaming();
// Check the Structures in allStructures to determine the next
// available name. The follow code uses a generation and ID number
// for an evolutionary/genetic algorithm. Other methods may only set
// the ID number. (note that the generation number is set already
// in the following example).
Structure *structure;
uint id = 1;
for (int j = 0; j < allStructures.size(); j++) {
  structure = allStructures.at(j);
  structure->lock()->lockForRead();
  if (structure->getGeneration() == generation &&
      structure->getIDNumber() >= id) {
    id = structure->getIDNumber() + 1;
  }
  structure->lock()->unlock();
}

// Assign data to structure (created elsewhere)
QWriteLocker locker (newStructure->lock());
newStructure->setIDNumber(id);
newStructure->setGeneration(generation);
newStructure->setParents(parents);
// Determine, create, and assign paths
QString id_s, gen_s, locpath_s, rempath_s;
id_s.sprintf("%05d",structure->getIDNumber());
gen_s.sprintf("%05d",structure->getGeneration());
locpath_s = filePath + "/" + gen_s + "x" + id_s + "/";
rempath_s = rempath + "/" + gen_s + "x" + id_s + "/";
QDir dir (locpath_s);
if (!dir.exists()) {
  if (!dir.mkpath(locpath_s)) {
    // Output error
  }
}
newStructure->setFileName(locpath_s);
newStructure->setRempath(rempath_s);
newStructure->setCurrentOptStep(1);
newStructure->findSpaceGroup();
// unlockForNaming(Structure*) unlocks the naming mutex and
// begins processing the structure that is passed.
m_queue->unlockForNaming(newStructure);

Most functions in this class do not need to be called as they are automatically called when needed. Check the source if in doubt.

Definition at line 94 of file queuemanager.h.

Constructor & Destructor Documentation

GlobalSearch::QueueManager::QueueManager	(	QThread *	thread,
		OptBase *	parent
	)			`[explicit]`

Constructor.

Parameters:

	thread	A QThread instance to run in
	parent	The OptBase class the QueueManager uses

Definition at line 61 of file queuemanager.cpp.

References m_thread, and moveToQMThread().

GlobalSearch::QueueManager::~QueueManager ( ) [virtual]

Destructor.

Definition at line 75 of file queuemanager.cpp.

References GlobalSearch::Structure::abortPreoptimization(), GlobalSearch::Tracker::list(), m_isDestroying, m_lastSubmissionTimeStamp, m_preOptTracker, and m_requestedStructures.

Member Function Documentation

void GlobalSearch::QueueManager::addManualStructureRequest ( int requests = 1 ) [slot]

Add a structure request. This must be called every time a structure is added via unlockForNaming(), and must be called first. This will not actually request the structure, it is only used for internal bookkeeping.

Note:: The mutex of m_tracker is locked while incrementing the request counter.

Definition at line 1166 of file queuemanager.cpp.

References GlobalSearch::Tracker::lockForWrite(), m_requestedStructures, m_tracker, and GlobalSearch::Tracker::unlock().

void GlobalSearch::QueueManager::addStructureToSubmissionQueue ( GlobalSearch::Structure * s ) [inline, protected, slot]

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts. Writes the input files for the optimization process and queues the Structure to be submitted for optimization at its current optStep.

Parameters:

s

The Structure to be submitted

Definition at line 296 of file queuemanager.h.

References addStructureToSubmissionQueue().

void GlobalSearch::QueueManager::addStructureToSubmissionQueue	(	GlobalSearch::Structure *	s,
		int	optStep
	)			`[protected, slot]`

Writes the input files for the optimization process and queues the Structure to be submitted for optimization.

Parameters:

	s	The Structure to be submitted
	optStep	The optStep to perform. s->currentOptStep is used if optStep==0.

Definition at line 885 of file queuemanager.cpp.

Referenced by addStructureToSubmissionQueue(), and senderHasFinishedPreoptimization().

void GlobalSearch::QueueManager::appendToJobStartTracker ( Structure * s ) [slot]

Appends a Structure to m_jobStartTracker. This should not be used unless resuming a session, and then only for structures that are marked Structure::WaitingForOptimization.

All other cases should use prepareStructureForSubmission(Structure*)

Parameters:

s

The Structure to be appended

See also:: prepareStructureForSubmission

Definition at line 1173 of file queuemanager.cpp.

References GlobalSearch::Tracker::append(), GlobalSearch::Tracker::lockForWrite(), m_jobStartTracker, and GlobalSearch::Tracker::unlock().

void GlobalSearch::QueueManager::checkLoop ( ) [protected, slot]

This is called automatically when the QueueManager is started. This function sets up a simple event loop that will run checkPopulation and checkRunning regularly.

Definition at line 199 of file queuemanager.cpp.

References checkPopulation(), checkRunning(), GlobalSearch::OptBase::isStarting, m_opt, m_thread, and GlobalSearch::OptBase::readOnly.

Referenced by setupConnections().

void GlobalSearch::QueueManager::checkPopulation ( ) [protected]

Check all Structures in the main Tracker and assign them to other trackers as needed (runningTracker, etc.).

If more structures are needed, they are requested in this function by emitting needNewStructure().

This function also submits new structures to the optimization engine if needed.

Also emits newStatusOverview for a summary of the queue's status.

See also:: newStatusOverview; needNewStructure

Definition at line 216 of file queuemanager.cpp.

Referenced by checkLoop().

void GlobalSearch::QueueManager::checkRunning ( ) [protected]

Monitors the Structures in getAllRunningStructures() and updates their statuses if they've changed.

Note:: Do note call this function directly; it is called automatically by the checkLoop function

Definition at line 373 of file queuemanager.cpp.

Referenced by checkLoop().

QList< Structure * > GlobalSearch::QueueManager::getAllDuplicateStructures ( ) [slot]

Returns:: All Structures in m_tracker with status Structure::Duplicate.

Definition at line 1050 of file queuemanager.cpp.

References GlobalSearch::Structure::Duplicate, GlobalSearch::Structure::getStatus(), GlobalSearch::Tracker::list(), GlobalSearch::Tracker::lockForRead(), m_tracker, and GlobalSearch::Tracker::unlock().

QList< Structure * > GlobalSearch::QueueManager::getAllOptimizedStructures ( ) [slot]

Returns:: All Structures in m_tracker with status Structure::Optimized.

Definition at line 1034 of file queuemanager.cpp.

References GlobalSearch::Structure::getStatus(), GlobalSearch::Tracker::list(), GlobalSearch::Tracker::lockForRead(), m_tracker, GlobalSearch::Structure::Optimized, and GlobalSearch::Tracker::unlock().

QList< Structure * > GlobalSearch::QueueManager::getAllPreoptimizingStructures ( ) [slot]

Returns:: All Structures in preOptTracker

Definition at line 1017 of file queuemanager.cpp.

References GlobalSearch::Tracker::list(), m_preOptTracker, and GlobalSearch::Tracker::rwLock().

QList< Structure * > GlobalSearch::QueueManager::getAllRunningStructures ( ) [slot]

Returns:: All Structures in m_runningTracker

Definition at line 1023 of file queuemanager.cpp.

References GlobalSearch::Tracker::list(), GlobalSearch::Tracker::lockForRead(), m_newStructureTracker, m_runningTracker, and GlobalSearch::Tracker::unlock().

Referenced by checkRunning().

QList< Structure * > GlobalSearch::QueueManager::getAllStructures ( ) [slot]

Returns:: All Structures in m_tracker and m_startPendingTracker

Definition at line 1066 of file queuemanager.cpp.

References GlobalSearch::Tracker::list(), GlobalSearch::Tracker::lockForRead(), m_newStructureTracker, m_tracker, and GlobalSearch::Tracker::unlock().

Referenced by lockForNaming().

void GlobalSearch::QueueManager::handleDuplicateStructure ( Structure * s ) [protected]

Perform actions on the Duplicate Structure s.

Parameters:

s

Structure of interest

Definition at line 778 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleEmptyStructure ( Structure * s ) [protected]

Perform actions on the Empty Structure s.

Parameters:

s

Structure of interest

Definition at line 609 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleErrorStructure ( Structure * s ) [protected]

Perform actions on the Error'd Structure s.

Parameters:

s

Structure of interest

Definition at line 624 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleInProcessStructure ( Structure * s ) [protected]

Perform actions on the InProcess Structure s.

Parameters:

s

Structure of interest

Definition at line 465 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleKilledStructure ( Structure * s ) [protected]

Perform actions on the Killed Structure s.

Parameters:

s

Structure of interest

Definition at line 739 of file queuemanager.cpp.

Referenced by checkRunning(), and handleRemovedStructure().

void GlobalSearch::QueueManager::handleOptimizedStructure ( Structure * s ) [protected]

Perform actions on the Optimized Structure s.

Parameters:

s

Structure of interest

Definition at line 511 of file queuemanager.cpp.

void GlobalSearch::QueueManager::handlePreoptimizingStructure ( Structure * s ) [protected]

Perform actions on the Preoptimizing Structure s.

Parameters:

s

Structure of interest

Definition at line 614 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleRemovedStructure ( Structure * s ) [protected]

Perform actions on the Removed Structure s.

Parameters:

s

Structure of interest

Definition at line 773 of file queuemanager.cpp.

References handleKilledStructure().

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleRestartStructure ( Structure * s ) [protected]

Perform actions on the Restart'ing Structure s.

Parameters:

s

Structure of interest

Definition at line 809 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleStepOptimizedStructure ( Structure * s ) [protected]

Perform actions on the StepOptimized Structure s.

Parameters:

s

Structure of interest

Definition at line 545 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleSubmittedStructure ( Structure * s ) [protected]

Perform actions on the Submitted Structure s.

Parameters:

s

Structure of interest

Definition at line 691 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleUpdatingStructure ( Structure * s ) [protected]

Perform actions on the Updating Structure s.

Parameters:

s

Structure of interest

Definition at line 619 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::handleWaitingForOptimizationStructure ( Structure * s ) [protected]

Perform actions on the WaitingForOptimization Structure s.

Parameters:

s

Structure of interest

Definition at line 604 of file queuemanager.cpp.

Referenced by checkRunning().

void GlobalSearch::QueueManager::killStructure ( Structure * s ) [slot]

Stops any running optimization processes associated with a structure and sets its status to Structure::Killed.

The structureKilled signal is emitted as well.

Parameters:

s

The Structure to kill.

See also:: structureKilled

Definition at line 858 of file queuemanager.cpp.

References GlobalSearch::Structure::getStatus(), GlobalSearch::Structure::Killed, GlobalSearch::Structure::Optimized, GlobalSearch::Structure::Removed, GlobalSearch::Structure::setStatus(), stopJob(), GlobalSearch::Structure::stopOptTimer(), and structureKilled().

QList< Structure * > GlobalSearch::QueueManager::lockForNaming ( ) [slot]

Locks the m_tracker and m_newStructure for reading.

Returns:: All Structures in getAllStructures()

Definition at line 1077 of file queuemanager.cpp.

References getAllStructures(), GlobalSearch::Tracker::lockForRead(), and m_tracker.

void GlobalSearch::QueueManager::movedToQMThread ( ) [signal]

Emitted when the QueueManager has been moved to it's final thread and is ready to accept connections.

Referenced by moveToQMThread().

void GlobalSearch::QueueManager::moveToQMThread ( ) [protected, slot]

Move this to the QThread specified in the constructor and setup connections in that thread's event loop.

Definition at line 131 of file queuemanager.cpp.

References m_thread, movedToQMThread(), and setupConnections().

Referenced by QueueManager().

void GlobalSearch::QueueManager::needNewStructure ( ) [signal]

Emitted when the number of unoptimized Structures drops below OptBase::contStructs. This is connected to OptBase::generateNewStructure() by default.

Referenced by checkPopulation(), and setupConnections().

void GlobalSearch::QueueManager::newStatusOverview	(	int	optimized,
		int	running,
		int	failing
	)			`[signal]`

Emitted when checkPopulation() is called to provide a short summary of the queuemanager's status.

Parameters:

	optimized	Number of optimized structures
	running	Number of running structures (e.g. submitted for optimization)
	failing	Number of structures with a getFailCount() > 0

Referenced by checkPopulation().

void GlobalSearch::QueueManager::reset ( ) [slot]

Reset all trackers in trackerList

Definition at line 169 of file queuemanager.cpp.

References m_jobStartTracker, m_needPreOptTracker, m_newStructureTracker, m_preOptTracker, m_runningTracker, and m_tracker.

Referenced by GlobalSearch::OptBase::reset().

void GlobalSearch::QueueManager::senderHasFinishedPreoptimization ( ) [protected, slot]

Add the emitter of the signal that triggers this slot to the submission queue

Definition at line 873 of file queuemanager.cpp.

References addStructureToSubmissionQueue(), m_preOptTracker, and GlobalSearch::Tracker::remove().

Referenced by startPreoptimization().

void GlobalSearch::QueueManager::setupConnections ( ) [protected, slot]

Called by moveToQMThread(), this function installs connections in the owning thread's event loop.

Definition at line 142 of file queuemanager.cpp.

References checkLoop(), m_opt, needNewStructure(), structureFinished(), structureKilled(), structureStarted(), structureSubmitted(), and structureUpdated().

Referenced by moveToQMThread().

void GlobalSearch::QueueManager::startJob ( ) [protected]

Submits the first Structure in m_jobStartTracker for submission. This should not be called directly, instead call prepareStructureForSubmission(Structure*).

See also:: prepareStructureForSubmission

Definition at line 962 of file queuemanager.cpp.

References GlobalSearch::Tracker::append(), GlobalSearch::Structure::Error, GlobalSearch::Structure::getCurrentOptStep(), GlobalSearch::Structure::getIDString(), GlobalSearch::Tracker::lockForWrite(), m_jobStartTracker, m_opt, m_preOptTracker, GlobalSearch::Structure::needsPreoptimization(), GlobalSearch::Tracker::popFirst(), GlobalSearch::OptBase::printBackTrace(), GlobalSearch::OptBase::queueInterface(), GlobalSearch::Structure::setStatus(), GlobalSearch::QueueInterface::startJob(), structureSubmitted(), GlobalSearch::Structure::Submitted, GlobalSearch::Tracker::unlock(), and GlobalSearch::OptBase::warning().

Referenced by checkPopulation().

void GlobalSearch::QueueManager::startPreoptimization ( ) [protected]

Submits the first Structure in m_needsPreOptTracker for preoptimization. This should not be called directly, the queuemanager will call this as appropriate.

See also:: Structure::setNeedsPreoptimization(); Structure::needsPreoptimization(); OptBase::preoptimizeStructure()

Definition at line 928 of file queuemanager.cpp.

References GlobalSearch::Tracker::append(), GlobalSearch::Tracker::lockForWrite(), m_needPreOptTracker, m_opt, m_preOptTracker, GlobalSearch::Structure::needsPreoptimization(), GlobalSearch::Tracker::popFirst(), GlobalSearch::OptBase::preoptimizeStructure(), GlobalSearch::Structure::Preoptimizing, senderHasFinishedPreoptimization(), GlobalSearch::Structure::setStatus(), and GlobalSearch::Tracker::unlock().

Referenced by checkPopulation().

void GlobalSearch::QueueManager::stopJob ( Structure * s ) [protected]

Kills the optimization process for the indicated Structure.

Parameters:

s

The Structure to stop optimizing.

Definition at line 1002 of file queuemanager.cpp.

References GlobalSearch::Structure::abortPreoptimization(), GlobalSearch::Structure::isPreoptimizing(), GlobalSearch::Tracker::lockForWrite(), m_jobStartTracker, m_opt, GlobalSearch::OptBase::queueInterface(), GlobalSearch::Tracker::remove(), GlobalSearch::QueueInterface::stopJob(), and GlobalSearch::Tracker::unlock().

Referenced by killStructure().

void GlobalSearch::QueueManager::structureFinished ( GlobalSearch::Structure * s ) [signal]

Emitted when a Structure has completed all optimization steps.

Parameters:

s

The Structure that has been updated

Referenced by setupConnections().

void GlobalSearch::QueueManager::structureKilled ( GlobalSearch::Structure * s ) [signal]

Emitted when a Structure has been killed through killStructure(Structure*)

Parameters:

s

The Structure that has been killed

See also:: killStructure

Referenced by killStructure(), and setupConnections().

void GlobalSearch::QueueManager::structureStarted ( GlobalSearch::Structure * s ) [signal]

Emitted when a Structure is accepted into the queuemanager (e.g. after submission through unlockForNaming(Structure*)

Parameters:

s

The Structure that has been accepted

See also:: lockForNaming; unlockForNaming

Referenced by setupConnections().

void GlobalSearch::QueueManager::structureSubmitted ( GlobalSearch::Structure * s ) [signal]

Emitted when a Structure is submitted for optimization

Parameters:

s

The Structure that has been submitted

Referenced by setupConnections(), and startJob().

void GlobalSearch::QueueManager::structureUpdated ( GlobalSearch::Structure * s ) [signal]

Emitted when a Structure has changed status. Useful for updating progress tables, plots, etc

Parameters:

s

The Structure that has been updated

Referenced by setupConnections().

void GlobalSearch::QueueManager::unlockForNaming ( Structure * s = 0 ) [slot]

Unlocks both the main and startPending trackers.

Parameters:

s

Optional new stucture to be added to the queuemanager and tracker.

Definition at line 1087 of file queuemanager.cpp.

References GlobalSearch::Tracker::append(), GlobalSearch::Structure::getIDString(), GlobalSearch::OptBase::isStarting, GlobalSearch::Tracker::lockForWrite(), m_isDestroying, m_newStructureTracker, m_opt, m_requestedStructures, m_tracker, and GlobalSearch::Tracker::unlock().

void GlobalSearch::QueueManager::updateQueue ( ) [protected]

Update the data from the remote PBS queue.

void GlobalSearch::QueueManager::updateStructure ( Structure * s ) [protected]

Called on Structures that are Structure::StepOptimized, this function will update the Structure with the results of the optimization and pass it along to prepareStructureForNextOptStep.

Parameters:

s

The step optimized structure