Wiki source code of 04. Technical information

Version 2.1 by Branislav ŠIŠKA on 2023/06/06 09:22

Show last authors
1 (% style="" %)
2 = {{id name="04.Technicalinformation-1.Hardware"/}}1. Hardware =
3
4 (% style="" %)
5 The Clindata software runs on computer cluster located on Institute of Molecular and Translational Medicine (IMTM), Faculty of Medicine and Dentistry, Palacky University in Olomouc. The facility is secured and under global surveillance.
6
7 (% style="" %)
8 === {{id name="04.Technicalinformation-Descriptionofhardware"/}}Description of hardware ===
9
10 (% style="" %)
11 **Servers**
12
13 (% style="" %)
14 HPE DL385 Gen10 CTO Mod-X 8SFF Svr
15
16 (% style="" %)
17 8x HPE 16GB 2Rx8 PC4-2933Y-R
18
19 (% style="" %)
20 2x HPE DL385 Gen10 AMD EPYC 7302
21
22 (% style="" %)
23 2x HPE 240GB SATA RI SFF SC DS SSD
24
25 (% style="" %)
26 **Data Storages**
27
28 (% style="" %)
29 HP 3PAR data storage 700TB.
30
31 (% style="" %)
32 HP EML tape library
33
34 (% style="" %)
35 Object storage
36
37 (% style="" %)
38 **Firewall**
39
40 (% style="" %)
41 HP F1000-S-EI VPN Firewall
42
43 (% style="" %)
44 **~ **
45
46 (% style="" %)
47 = {{id name="04.Technicalinformation-2.Software"/}}2. Software =
48
49 (% style="" %)
50 === {{id name="04.Technicalinformation-Thesoftwarerequirements"/}}The software requirements ===
51
52 (% style="" %)
53 The only requirement for using the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is Internet browser which supports HTML5 standard. The list of supported browsers:
54
55 (% style="" %)
56 * Chrome: (Current - 1) and Current
57 * Edge: (Current - 1) and Current
58 * Firefox: (Current - 1) and Current
59 * Internet Explorer: 11+
60 * Safari: (Current - 1) and Current
61 * Opera: Current
62
63 (% style="" %)
64 (Current means the last available version of given browser)
65
66 (% style="" %)
67 === {{id name="04.Technicalinformation-Programminglanguage"/}}Programming language ===
68
69 (% style="" %)
70 The main programming language used for development of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) application is Java 8. Other technologies used for development are:
71
72 (% style="" %)
73 * Spring Framework v5.
74 * HTML, CSS
75 * JavaScript
76 * jQuery
77 * jQuery UI
78 * Bootstrap
79 * MathJS
80 * Datatables
81 * SQL
82 * Oracle database
83
84 (% style="" %)
85 === {{id name="04.Technicalinformation-Operationsystem"/}}Operation system ===
86
87 (% style="" %)
88 Operation system installed on production servers is **RedHat Enterprise Linux 7.4.**
89
90 (% style="" %)
91 === {{id name="04.Technicalinformation-Proxyserver"/}}Proxy server ===
92
93 (% style="" %)
94 The **Apache HTTP Server** is used as gateway from outside world to internal application running in the production server.
95
96 (% style="" %)
97 === {{id name="04.Technicalinformation-Applicationserver"/}}Application server ===
98
99 (% style="" %)
100 The **ClinData** application runs on **Apache Tomcat,** which is an open-source Java Servlet Container developed by the Apache Software Foundation
101
102 (% style="" %)
103 **~ **
104
105 (% style="" %)
106 = {{id name="04.Technicalinformation-3.Database"/}}3. Database =
107
108 (% style="" %)
109 === {{id name="04.Technicalinformation-TheClindataDatabase"/}}The (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) Database ===
110
111 (% style="" %)
112 The database used for storing data from the (% style="color: rgb(23,43,77);text-decoration: none;" %)**Clindata**(%%)** **software is **Oracle Database** (commonly referred to as **Oracle RDBMS**) which is produced by Oracle Corporation. Version of database is 12.1. Standard edition.
113
114 (% style="" %)
115 The Oracle database runs on separated Linux based server which si firewalled from external network (Internet) by hardware firewall. This database server is not accessible from outside of organization but only from enlisted inner servers (application and backup servers).
116
117 (% style="" %)
118 **~ **
119
120 (% style="" %)
121 = {{id name="04.Technicalinformation-4.Backup"/}}4. Backup =
122
123 (% style="" %)
124 There are more levels of data archiving to ensure data safety and quick database recovery. Data are archived on **database level** and **operation system level**
125
126 (% style="" %)
127 1. Database level backups
128 1*. **RMAN** utility is integral part of the Oracle database. It creates binary copy of whole database and stores it to filesystem. The RMAN utility is run **every week**. The files are stored internally on database server and are copied to two independent backup sites.
129 1*. **EXPDP/IMPDP** is data pump exporting data into text base backups. The EXPDP utility is run **every 4 hours**. The backup target is the same as with RMAN. It is stored to two independent backup sites.
130 1*. **Redo Logs** are archived **every day** to filesystem.
131 1. Operation system backups
132
133 (% style="" %)
134 * **IBM Tivoli Storage Manager** (TSM Admin) is enterprise solution from IBM for backups and recovery of physical or virtual servers. The backup created by TSM Admin includes redo logs, RMAN and EXPDP exports. It runs **every day** and the backup data is stored to disk array.
135
136 (% style="" %)
137 RMAN configuration file
138
139 (% style="" %)
140 CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
141 CONFIGURE BACKUP OPTIMIZATION OFF; # default
142 CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
143 CONFIGURE CONTROLFILE AUTOBACKUP ON;
144 CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
145 CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
146 CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
147 CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
148 CONFIGURE MAXSETSIZE TO UNLIMITED; # default
149 CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
150 CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
151 CONFIGURE RMAN OUTPUT TO KEEP FOR 7 DAYS; # default
152 CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
153 CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/../../oracle/12c/dbs/snapcf_imtm.f'; # default
154
155 (% style="" %)
156 EXPDP configuration file
157
158 (% style="" %)
159 DIRECTORY=dtpump
160 DUMPFILE=registry.dmp
161 LOGFILE=registry.log
162 CONTENT=ALL
163 COMPRESSION=NONE
164 JOB_NAME=registry_migration
165 SCHEMAS=registry,registry_aud
166
167 (% style="" %)
168 = {{id name="04.Technicalinformation-05.Secureconnection"/}}05. Secure connection =
169
170 (% style="" %)
171 === {{id name="04.Technicalinformation-Security"/}}Security ===
172
173 (% style="" %)
174 As the (% style="color: rgb(23,43,77);text-decoration: none;" %)**Clindata**(%%)** **application is **web-based** application there is need to **secure communication** between **server** and **client's computer**. It is done by using **HTTPS** communication protocol which is encrypted using Transport Layer Security **(TLS).** This protocol is widely used for all secure transactions on the Internet (payment, emails etc.) and is considered as safe and unbreakable. It protects against man in-the middle attacks. Communication without the security layer (HTTP) is can be interfered by attackers, they can listen to it or change it.
175
176 (% style="" %)
177 === {{id name="04.Technicalinformation-Securityredirection"/}}Security redirection ===
178
179 (% style="" %)
180 All user requests coming via unsecured **HTTP** protocol are automatically **redirected** to secure **HTTPS** protocol. All communication between client and server is secured and there is no way how to connect to the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software via unsecured connection.
181
182 (% style="" %)
183 === {{id name="04.Technicalinformation-Certificate"/}}Certificate ===
184
185 (% style="" %)
186 The secured communication requires a certificate stored on the web server. The certificate must be signed by **trusted certificate authority**. The (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) server uses the certificate digitally signed by **TERENA** authority.
187
188 (% style="" %)
189 **~ **
190
191 (% style="" %)
192 = {{id name="04.Technicalinformation-06.Authenticationandauthorization"/}}06. Authentication and authorization =
193
194 (% style="" %)
195 === {{id name="04.Technicalinformation-Usersadministration"/}}Users administration ===
196
197 (% style="" %)
198 All user using the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) application must be registered before they can log in. There is no possibility to get unauthorized access to the server even for some demonstration purposes. There is a specialized application for user management - IMTM Admin tool.
199
200 (% style="" %)
201 The **Admin tool** is responsible for:
202
203 (% style="" %)
204 * Management of **institutions, companies, hospitals** and their departments. There can be unlimited number of organization levels, for example a university can have such structure university-faculty-department-laboratory. Each organization level can obtain different set of privileges and roles.
205 * Management of **users**. Every user is identified by email address as login and password. Users are assigned to their organizations. Users can work on more projects with different roles. It is allowed by** user profiles**. Number of profiles for a user is not limited. Each profile can have different set of **privileges and roles**.
206 * Management **roles and profiles**.
207
208 (% style="" %)
209 The Admin database with user's data is stored in the Oracle database as separated schema. Access to this schema is restricted only for admin users. The server with the Oracle database is firewalled out of public network and not accessible from Internet.
210
211 (% style="" %)
212 An account for new user can be created **only by administrator**. There is no way that user could create his account on its own.
213
214 (% style="" %)
215 These steps must be followed to **create new account**:
216
217 (% style="" %)
218 * New user asks a project owner to create new account
219 * The project owner asks administrator to create new account with specified privileges and roles
220 * The administrator creates new account and sets required privileges and roles
221 * The project owner checks account setting and approve it.
222 * New user receives his credentials and can log in.
223
224 (% style="" %)
225 === {{id name="04.Technicalinformation-Centralauthenticationservice(CAS)"/}}Central authentication service (CAS) ===
226
227 (% style="" %)
228 The (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) application must be connected with data from the IMTM Admin to control accounts, roles and privileges. It is done by integrating of the CAS technology into the ClinData software. The CAS technology consists of CAS Server and CAS Client.
229
230 (% style="" %)
231 The CAS server is responsible for authenticating users and granting accesses to applications. The CAS clients protect the CAS applications and retrieve the identity of the granted users from the CAS server.
232
233 (% style="" %)
234 = {{id name="04.Technicalinformation-07.PrivilegesandRoles"/}}07. Privileges and Roles =
235
236 (% style="" %)
237 === {{id name="04.Technicalinformation-Accessrestrictions"/}}**Access restrictions** ===
238
239 (% style="" %)
240 A user access can be restricted in two different areas:
241
242 (% style="" %)
243 * restriction in **access to** ClinData **functionality**
244 * restriction in **access to data** stored in the ClinData software
245
246 (% style="" %)
247 All restriction is set in the **IMTM Admin tool**.
248
249 (% style="" %)
250 === {{id name="04.Technicalinformation-Functionalityrestrictions"/}}Functionality restrictions ===
251
252 (% style="" %)
253 **Privileges**
254
255 (% style="" %)
256 Access privileges determine which (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) objects a user can browse or edit. Each functionality in the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is reflected in corresponding privilege so the access to everything is controlled. Any user or group of users can have access to any privilege granted or restricted.
257
258 (% style="" %)
259 The picture shows schema of privileges in the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software.
260
261 (% style="" %)
262 **Roles**
263
264 (% style="" %)
265 Roles are virtual entities which serve as container for more privileges.
266
267 (% style="" %)
268 There are predefined roles and users, or groups of users can be assigned to them. The most frequently used roles are:
269
270 (% style="" %)
271 * ClinData system admin - full access to all functions in ClinData, no restrictions, creating new project
272 * ClinData project admin - full access to all function in selected project including study designer
273 * ClinData project data manager - access to all functions needed to insert new/update patient’s data.
274 * ClinData project data monitor - access to all functions needed for study monitoring, validation and finishing CRFs.
275 * ClinData project data browser - read only access to selected data.
276
277 (% style="" %)
278 \\
279
280 (% style="" %)
281 === {{id name="04.Technicalinformation-Datarestrictions"/}}Data restrictions ===
282
283 (% style="" %)
284 Default setting for accessing of data in the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is maximally restricted. A user can see only data he inserted himself. By default, he doesn't see any data inserted by any other user. Access to any other data must be explicitly permitted.
285
286 (% style="" %)
287 These options can be set:
288
289 (% style="" %)
290 * user can see only his data
291 * user can see data inserted by other user or group of users
292 * user can see data linked with an organization
293 * user can see all data in a study
294
295 (% style="" %)
296 === {{id name="04.Technicalinformation-Personaldata"/}}Personal data ===
297
298 (% style="" %)
299 There can be studies or registers which contain personal data. Access to this data can be restricted by special privilege.
300
301 (% style="" %)
302 These options can be set:
303
304 (% style="" %)
305 * user can see personal data
306 * user can't see personal data
307
308 (% style="" %)
309 \\
310
311 (% style="" %)
312 = {{id name="04.Technicalinformation-8.Logging"/}}8. Logging =
313
314 (% style="" %)
315 The ClinData software **records everything** happening in the system. Admin user can browse these records in user friendly way and analyze potential problems, watch user activities etc.
316
317 (% style="" %)
318 There are three different types of logging mechanisms:
319
320 (% style="" %)
321 * **Software logging** is done on programming language level and is very detailed. The log files contain data about internal state of the whole system in time of logging event. This approach is designed for detailed analyses of problems which happened in past.
322 * **Access logging** is designed for controlling of user’s activities. The access record contains data about who did an action and when. It logs all actions done on all objects in the system. Object can be study, patient, CRF form, file. These actions are logged:
323 ** create
324 ** open
325 ** change
326 ** add
327 ** remove
328 ** delete
329 ** export
330 * **Auditing** is focused to changes done in CRF forms. It records complete history of what was changed by users. One record contains data about:
331 ** when the change was done
332 ** who changed the data
333 ** what was changed
334 ** what is the new value
335
336 (% style="" %)
337 The important information is that the ClinData software** doesn't delete any record**. Every record in the database has system **flag ACTIVE**. Deleting of the row just sets this **ACTIVE** flag to **false**. The inactive rows are not displayed in the ClinData software but are still stored in the database.
338
339 (% style="" %)
340 **~ **
341
342 (% style="" %)
343 = {{id name="04.Technicalinformation-9.Softwaredevelopment"/}}9. Software development =
344
345 (% style="" %)
346 === {{id name="04.Technicalinformation-Issuetracking"/}}Issue tracking ===
347
348 (% style="" %)
349 Any problem found in the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is documented and created as a **new issue in JIRA software**. JIRA software is developed by Atlassian and is an issue tracking tool. The new issue is analyzed, and priority is assigned.  The list of issues is sorted by priorities and processed by developers. When a serious problem is fixed then it is published in new version of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software. The issue is also closed as done in JIRA.
350
351 (% style="" %)
352 === {{id name="04.Technicalinformation-Changesmanagement"/}}Changes management ===
353
354 (% style="" %)
355 All requests for changes planned in the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software are stored in JIRA. When a new request is coming then it is analyzed, time estimation is done, and priority assigned. The list of issues is sorted by priorities and processed by developers.
356
357 (% style="" %)
358 === {{id name="04.Technicalinformation-Versioning"/}}Versioning ===
359
360 (% style="" %)
361 The source code of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is stored in** GIT repository** which allows tracking of changes in files. There is possibility to browse history of any source code file in the repository. Every change is also documented so it is easy to understand the development cycle.
362
363 (% style="" %)
364 === {{id name="04.Technicalinformation-Codereview"/}}Code review ===
365
366 (% style="" %)
367 Any change done in source code of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software must be **reviewed** by another developer. This process is called **code review**. This process minimizes number of bugs in source code because everything is double checked. **Bitbucket software** (developed by Atlassian) is used for code reviews. It prevents developers from using not proven code in public versions of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software.
368
369 (% style="" %)
370 **~ **
371
372 (% style="" %)
373 = {{id name="04.Technicalinformation-10.Qualityassurance"/}}10. Quality assurance =
374
375 (% style="" %)
376 === {{id name="04.Technicalinformation-Testingenvironment"/}}Testing environment ===
377
378 (% style="" %)
379 All new versions of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software must be tested and proven as functional and correct before they are published. There is a special environment which is used form testing of the new version before it is published. The testing environment must be similar to production environment to avoid configuration issues.
380
381 (% style="" %)
382 === {{id name="04.Technicalinformation-Unittesting"/}}Unit testing ===
383
384 (% style="" %)
385 Unit testing is a software testing method by which individual units of source code are tested to determine whether they are fit for use. There are actually more than one thousand-unit tests in the source code of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software. All critical parts of the source code are covered by unit test close to 100%. Overall source code is covered by unit test by more than 85%. Any problem in unit testing is blocker for publishing of the version of the software.
386
387 (% style="" %)
388 === {{id name="04.Technicalinformation-Applicationtesting"/}}Application testing ===
389
390 (% style="" %)
391 The whole application is tested by application exploratory testing before it is published. The application testing is done in testing environment. Any problem in application testing is blocker for publishing of the version.
392
393 (% style="" %)
394 === {{id name="04.Technicalinformation-Publishing"/}}Publishing ===
395
396 (% style="" %)
397 Publishing process means that a new version of the (% style="color: rgb(23,43,77);text-decoration: none;" %)Clindata(%%) software is being released and made accessible for users. The Bamboo software (developed by Atlassian) is used for building and publishing new versions. Unit testing is also involved in publishing of the new version. In case of any problem in any unit test the whole publishing, process is interrupted, and an notification email is sent to responsible persons.