The SemanticSegmentation class is used to create binary masks for the reconstruction. The masks are based on a semantic map created by an neural network model. The models can be added to the models/SemanticSegmentation path. To be recognized and loaded by the plugin they need to be in .onnx format. Plugin allows to use CUDA api with cuDNN to accelerate evaluation of the model using the gpu. More...

#include <semanticsegmentation.h>

Inheritance diagram for SemanticSegmentation:

Classes
struct	ClassInfo

struct	ModelInfo

Signals
void	sig_classesAndColorsChanged (QStringList classes, QColorList colors, QBoolList selectedClasses)
	[signal] sig_classesAndColorsChanged is emitted if a new model with different classes and colors is selected.

void	sig_message (QString processor, QString message="", bool active=true)
	[signal] sig_message is emmitted if the algorithm has a message for the user to display.

Signals inherited from ITransform
void	sendToGui (uint idx, const cv::Mat &img)
	[signal] sendToGui is emitted if the ITransform instance has an image to display to the user.

Public Member Functions
	SemanticSegmentation ()
	Creates a Semantic Segmentation Plugin and loads the models for image segmentation.

QWidget *	getSettingsWidget (QWidget *parent) override
	getSettingsWidget provides a QWidget for the user. This allows to select one of the models for segmentation. The classes with their associated colors are listed and the ones to include in the output mask can bee selected. The blending alpha for overlaying the semantic map can be adjusted.

QString	getName () const override
	getName returns a name for displaying this algorithm to the user.

ITransform *	copy () override
	copy creates a new SemanticSegmentation instance and copies the attributes from this.

TransformResult	transform (uint idx, const cv::Mat &img, const Resolution &resolution, const ROI &roi) override
	transform creates a binary mask for the given image. A nn model is loaded from an .onnx file and used to compute a semantic map. From this a binary mask for the image is computed and returned. The model as well as the classes to include in the mask can be selected in the SettingsWidget accessable through SemanticSegmentation::getSettingsWidget. The ITransform::sendToGui signal is emitted before the calculation is started to present the given image to the user. The gui is once more updated with the finished semantic map and mask side by side.

void	enableCuda (bool enabled) override
	enableCuda is called if the user toggles the use CUDA flag in the core application.

void	setSettings (QMap< QString, QVariant > settings) override
	setter for plugin's settings

QMap< QString, QVariant >	getSettings () override
	getter for plugin's settings

void	deactivate () override
	deactivate will be called when the plugin is no longer used, i.e. when the user deselects it in the sampling window or when exporting finishes. This should be used to free resources such as gpu memory.

Public Member Functions inherited from ITransform
virtual void	activate ()
	activate will be called before the plugin is used, i.e. when the user selects it in the seampling window or when exporting.

Detailed Description

The SemanticSegmentation class is used to create binary masks for the reconstruction. The masks are based on a semantic map created by an neural network model. The models can be added to the models/SemanticSegmentation path. To be recognized and loaded by the plugin they need to be in .onnx format. Plugin allows to use CUDA api with cuDNN to accelerate evaluation of the model using the gpu.

Author: Dominik Wüst

Date: 2021/03/10

Member Function Documentation

◆ copy()

ITransform * SemanticSegmentation::copy ( )

overridevirtual

copy creates a new SemanticSegmentation instance and copies the attributes from this.

Returns: A pointer to the new instance.

Implements ITransform.

◆ enableCuda()

void SemanticSegmentation::enableCuda ( bool enabled )

overridevirtual

enableCuda is called if the user toggles the use CUDA flag in the core application.

Parameters

enabled allows to ue CUDA api if true

Implements ITransform.

◆ getName()

QString SemanticSegmentation::getName ( ) const

overridevirtual

getName returns a name for displaying this algorithm to the user.

Returns: Semantic Segmentation Algorithm as QString.

Implements ITransform.

◆ getSettings()

QMap< QString, QVariant > SemanticSegmentation::getSettings ( )

overridevirtual

getter for plugin's settings

Returns: QMap with the settings

Implements ITransform.

◆ getSettingsWidget()

QWidget * SemanticSegmentation::getSettingsWidget ( QWidget * parent )

overridevirtual

getSettingsWidget provides a QWidget for the user. This allows to select one of the models for segmentation. The classes with their associated colors are listed and the ones to include in the output mask can bee selected. The blending alpha for overlaying the semantic map can be adjusted.

Parameters

parent The parent for the QWidget

Returns: The QWidget with the settings

Implements ITransform.

◆ setSettings()

void SemanticSegmentation::setSettings ( QMap< QString, QVariant > settings )

overridevirtual

setter for plugin's settings

Parameters

QMap	with the settings

Implements ITransform.

◆ sig_classesAndColorsChanged

void SemanticSegmentation::sig_classesAndColorsChanged	(	QStringList	classes,
		QColorList	colors,
		QBoolList	selectedClasses
	)

signal

[signal] sig_classesAndColorsChanged is emitted if a new model with different classes and colors is selected.

Parameters

classes	The names of the new classes
colors	The colors in the semantic map for the new classes
selectedClasses	bools for the currently selected classes

◆ sig_message

void SemanticSegmentation::sig_message	(	QString	processor,
		QString	message = `""`,
		bool	active = `true`
	)

signal

[signal] sig_message is emmitted if the algorithm has a message for the user to display.

Parameters

processor	The processor currently used (cpu or gpu)
message	The information to display as text
active	If the process is still active (true) or now idle (false)

◆ transform()

TransformResult SemanticSegmentation::transform	(	uint	idx,
		const cv::Mat &	img,
		const Resolution &	resolution,
		const ROI &	roi
	)

overridevirtual

transform creates a binary mask for the given image. A nn model is loaded from an .onnx file and used to compute a semantic map. From this a binary mask for the image is computed and returned. The model as well as the classes to include in the mask can be selected in the SettingsWidget accessable through SemanticSegmentation::getSettingsWidget. The ITransform::sendToGui signal is emitted before the calculation is started to present the given image to the user. The gui is once more updated with the finished semantic map and mask side by side.

Parameters

idx	The index of the image to transform
img	The image to transform

Returns: A list containing the binary mask, or empty if no model selected

See also: ITransform::sendToGui

Implements ITransform.

The documentation for this class was generated from the following files:

iVS3D/src/iVS3D-semanticSegmentationPlugin/semanticsegmentation.h
iVS3D/src/iVS3D-semanticSegmentationPlugin/semanticsegmentation.cpp

Classes

Signals

Public Member Functions

Detailed Description

Member Function Documentation

◆ copy()

◆ enableCuda()

◆ getName()

◆ getSettings()

◆ getSettingsWidget()

◆ setSettings()

◆ sig_classesAndColorsChanged

◆ sig_message

◆ transform()