Li Guoliang's Blog

Jul 9, 2019

Relaxed Binding of Type-safe Configuration Properties in Spring Boot 1.5

Question: what’s the output?

Given this application.yml:

freeMarker: # with upper-case M
  templatePath: "/freeMarker/upper-case"
 
freemarker:
  templatePath: "/freemarker/lower-case"

and this configuration bean:

@Component
@ConfigurationProperties(prefix = "freeMarker")
public class FreeMarkerConfigurationSettings {
 
    String templatePath;
 
    public String getTemplatePath() {
        return templatePath;
    }
 
    public void setTemplatePath(String templatePath) {
        this.templatePath = templatePath;
    }
}

what’s the output of this snippet?

@Component
public class PropertiesPrinter implements CommandLineRunner {
 
    @Autowired
    FreeMarkerConfigurationSettings freeMarkerConfigurationSettings;
 
    @Override
    public void run(String... strings) throws Exception {
        System.out.println("templatePath using ConfigurationProperties: " + freeMarkerConfigurationSettings.templatePath);
    }
}

looks like the property should be resolved to “/freeMarker/upper-case”, but actually it’s not. the output is:

templatePath using ConfigurationProperties: /freemarker/lower-case

What? I found a bug in Spring Boot?

No. actually this is a feature: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-relaxed-binding

Spring Boot uses some relaxed rules for binding Environment properties to @ConfigurationProperties beans, so there does not need to be an exact match between the Environment property name and the bean property name. Common examples where this is useful include dash-separated environment properties (for example, context-path binds to contextPath), and capitalized environment properties (for example, PORT binds to port).

ok, it’s a feature, then how does Spring boot choose the one? (from a list of candidates?)

I didn’t find documents with concrete details, so I digged into the source code. Spring did the magic during property binding which is part of the ‘auto-wired’ initialisation: getPropertyValuesForNamePrefix finds the final property value from a list of candidates with a list of “RelaxedNames”, i.e the 2 “propertyValues” we defined in the application.yml:

private MutablePropertyValues getPropertyValuesForNamePrefix(
      MutablePropertyValues propertyValues) {
   if (!StringUtils.hasText(this.namePrefix) && !this.ignoreNestedProperties) {
      return propertyValues;
   }
   MutablePropertyValues rtn = new MutablePropertyValues();
   for (PropertyValue value : propertyValues.getPropertyValues()) {
      String name = value.getName();
      for (String prefix : new RelaxedNames(stripLastDot(this.namePrefix))) {
         for (String separator : new String[] { ".", "_" }) {
            String candidate = (StringUtils.hasLength(prefix) ? prefix + separator
                  : prefix);
            if (name.startsWith(candidate)) {
               name = name.substring(candidate.length());
               if (!(this.ignoreNestedProperties && name.contains("."))) {
                  PropertyOrigin propertyOrigin = OriginCapablePropertyValue
                        .getOrigin(value);
                  rtn.addPropertyValue(new OriginCapablePropertyValue(name,
                        value.getValue(), propertyOrigin));
               }
            }
         }
      }
   }
   return rtn;
}

what is the “RelaxedNames” for “freeMarker” we defined in our configuration bean?

values = {LinkedHashSet@3080} size = 7
= "freeMarker"
= "free_marker"
= "free-marker"
= "freemarker"
= "FREEMARKER"
= "FREE_MARKER"
= "FREE-MARKER"

I would say that “rtn.addPropertyValue()” is not a good name:

/**
 * Add a PropertyValue object, replacing any existing one for the
 * corresponding property or getting merged with it (if applicable).
 * @param pv PropertyValue object to add
 * @return this in order to allow for adding multiple property values in a chain
 */
public MutablePropertyValues addPropertyValue(PropertyValue pv) {
   for (int i = 0; i < this.propertyValueList.size(); i++) {
      PropertyValue currentPv = this.propertyValueList.get(i);
      if (currentPv.getName().equals(pv.getName())) {
         pv = mergeIfRequired(pv, currentPv);
         setPropertyValueAt(pv, i);
         return this;
      }
   }
   this.propertyValueList.add(pv);
   return this;
}

“Add a PropertyValue object, replacing any existing one for the corresponding property or getting merged with it (if applicable).”

Spring boot resolves the property’s value by comparing each properties with a list of “RelaxedNames” and the return the last one it found.

Conclusions

Relaxed binding is a nice feature, but it could confuse you with the resolved value: spring boot will return the last found property’s value based on the list of “RelaxedNames”
Choose better “prefix” names. e.g. “freemarker” is too general, “obsFreemarker” is better.

Nov 7, 2018
在新加坡申请泰国旅游签证

虽然拿中国护照可以在泰国”落地签”, 但是由于家里老人没有单独去其他国家的经历, 加上担心落地签排队, 决定在新加坡的大使馆直接申请一张旅游签证.

跟其他国家/地区类似, 泰国驻新加坡大使馆也只是负责新加坡居民的签证服务. 譬如EP,PR,LTPV等. 具体信息及流程:https://visaonline.thaiembassy.sg/index.php

这里单说官方没有提到的细节:
- 网上填资料申请时可以上传照片
- 去现场交资料时只需要按照要求打印申请表, 并不需要额外提供照片
- 去现场交资料时, 家人可以陪着进去
- 其他人可以代领护照
时间表:
- 04-11-2018 23:38:54 visaonline提交资料: 照片(手机拍的) + 护照首页照片 + 机票 + LTPV正反面 + 银行对账单
- 05-11-2018 傍晚, visaonline状态改为”reviewed”, 提示可以在Nov 6/7去交资料(打印并签名的申请表x2 + 机票等辅助资料复印件x1)
- 06-11-2018 新加坡假期放假
- 07-11-2018 现场提交资料 + 护照, 被告知”collect passort with the receipt tomorrow afternoon 2pm - 3pm” 下午visaonline状态改为”to collect passport”

Aug 18, 2018

Run Selenium with Headless Chrome in Docker(CentOS)

As an engineer, I want to test my web app with Chrome in a Jenkins cluster. Chrome is not available in most of Jenkins build nodes, but docker is.

build image

based on CentOS, install headless chrome, selenium and chromedriver.

FROM centos:7

LABEL org.label-schema.schema-version="1.0" \
    org.label-schema.name="Selenium with Headless Chrome and CentOS" \
    org.label-schema.vendor="liguoliang.com" \
    org.label-schema.license="GPLv2" \
    org.label-schema.build-date="20180817"

# install necessary tools
RUN yum install unzip -y
RUN curl -O https://bootstrap.pypa.io/get-pip.py
RUN python get-pip.py

# install headless chrome
RUN curl -O  https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
RUN yum install google-chrome-stable_current_x86_64.rpm -y

# install selenium
RUN pip install selenium

# download chromedriver
RUN mkdir /opt/chrome
RUN curl -O https://chromedriver.storage.googleapis.com/2.41/chromedriver_linux64.zip
RUN unzip chromedriver_linux64.zip -d /opt/chrome

# copy the testing python script
COPY selenium-with-headless-chrome.py .
RUN python selenium-with-headless-chrome.py

execute selenium from python

from selenium import webdriver
from selenium.webdriver.chrome.options import Options


chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--no-sandbox")

driver = webdriver.Chrome(r'/opt/chrome/chromedriver', chrome_options=chrome_options)

driver.get("http://python.org")
print "page title: %s" % driver.title

docker image & source code

docker image: https://hub.docker.com/r/guoliangli/selenium-with-headless-chrome-centos/
source: https://github.com/guoliang-dev/docker-selenium-with-headless-chrome-centos

troubleshootings

Public key for google-chrome-unstable-x.rpm is not installed

solution: import google key e.g.

rpm --import https://dl-ssl.google.com/linux/linux_signing_key.pub

for more detials: https://www.google.com/linuxrepositories/

ERROR Message: unknown error: DevToolsActivePort file doesn’t exist

add arguments:

 "--headless",
 "--no-sandbox",
 "--disable-dev-shm-usage"

for more details:

https://github.com/timgrossmann/InstaPy/issues/2362
https://github.com/karma-runner/karma-chrome-launcher/issues/158

Aug 11, 2018
'Software Testing Foundations' Notes

randomly copied from: Software Testing Foundations, 4th Edition

The Fundamental Test Process

Test Planing and Control

Planning of the test process starts at the beginning of the software development project. The mission and objectives of testing must be defined and agreeed upon as well as the resources necessary for the test process.

The main task of planning is to determine the test strategy or approach. since an exhaustive test is not possible, priorities must be set based on risk assessment. The test activities must be distributed to the individual subsystems, denpending on the expected risk and the sverity of failure effects.

Test Analysis and Design

The first task is to review the test basis, i.e., the specification of what should be tested. the specification should be concrete and clear enough to develop test cases. the basis for the creation of a test can be the specification or architecure documents.

it is important to ensure traceability between the specifications to be tested and the tests themselves. it must be clear which test cases test which requirement and vice versa. Only this way is it possible to decide which requirements are to be or have been tested, how intensively and with which test cases.Even the traceability of requirement changes to the test cases and vice versa should be verified.

Test Implementation and Execution

Tests must be run and logged. the priority of the test cases decided during planning.

Test Evaluation and Reporting

During test evaluation and reporting, the test object is assessed against the set test exit criteria specified during planning. this ma result in normal termination of the tests if all criteria are met, or it may be decided that additional test cases should be run or that the criteria weree too hard. it must be decided whether the test exit criteria defined in the test plan are fulfilled.

Test Closure Activities

the following data should be recorded:
- when wwas the software system released?
- when was the test finished or terminated?
- when was a milestone reached or a maintenane release completed? Importeant informantion for evaluation can be extracted by asking the following questions:
- which planned results are achieved and when – if at all?
- which unexpected events happened (reasons and how they were met)?
- are there any open problems? and change requests? why ere thye not implemented?
- how was user acceptance after deploying the system?
General Principles of Testing
- Testing shows the presence of defects, not their absence. Testing can show that the product fails, cannot prove that a program is defect free. even if no failures are found during testing, this is no proof that there are no defects.
- Exhaustive tesing is impossible it’s impossible to run an exhaustive test that includes all possible values.
- Testing activities should start as early as possible
- Defect clustering. if many defects are detected in one place, there are normally more defects nearby.
- The pesticide paradox. new and modified cases should be developed and added to the test.
- Testing is context dependent.
- No failures means the system is useful is a fallacy.
Test Plan

The test manager might participate in the following planning activities:
- defininingthe overall approach to and strategy for testing
- deciding about the test environment and test auotomation
- defining the test level and their interaction and integrating the testing activities with other project activities
- deciding how to evaluate the test results
- selecting matrics for monitoring and controlling test work, as well as defining test exit criteria
- determining how much test documentation shall be prepared and determining templates
- writing the test plan and deciding on what, who, when and how much testing
- estimating test effor and test cost.
Test Entry and Exit Criteria

typial entry criteria:
- the test environment is ready
- the test tools are ready for use in the test environment
- test objects are installed in the test environment
- the necessary test data is available
exit:
- achieved test coverage: tests run, covered requirements, code coverage etc.
- product quality: defect density, defect severity, failure rate and reliability of the test object
- residual risk: tests not executed, defects not repaired, incomplete coverage of requirements or code. ect.
- economic constraints: allowed cost, project risks, release deadlines and market chance.
Test Plan according to IEEE 829-1998
- Test Plan Identifier
- Introduction
- Test Items
- Features to be Tested
- Features not to be Tested
- Approach
- Item Pass/Fail Criteria (exit criteria)
- Suspension Criteria and Resumption Requirements
- Test Deliverables
- Testing Tasks
- Environmental Needs
- Staffing and Training Needs
- Schedule
- Risks and Contingencies
- Approvals
Jul 4, 2018
npm versioning

npm versioning

https://docs.npmjs.com/getting-started/semantic-versioning

[major, minor, patch]

e.g.
- fist release: 1.0.0
- bug fix, minor change -> 1.0.1, 1.0.2
- non-breaking new features -> 1.1.0
- breaking changes -> 2.0.0
Semver for Consumers
- different ways to define semantic versiong
- npm semver calculator
update dependencies based on semantic versions

npm install:

npm will look at the dependencies that are listed in that file and download the latest versions, using semantic versioning. (https://docs.npmjs.com/getting-started/using-a-package.json#managing-dependency-versions)

yarn upgrade:

Upgrades packages to their latest version based on the specified range. (https://yarnpkg.com/lang/en/docs/cli/upgrade/)

prerelease tags and ranges

based on the following check:
```
const semver = require('semver')
> semver.gt('1.1.1-rc.1', '1.1.0')
true
> semver.gt('1.1.2-rc.1', '1.1.1-rc.1')
true
> semver.gt('1.1.2-rc.2', '1.1.2-rc.1')
true
```
I would assume that given a list of versions: ['1.1.0', '1.1.1-rc.3', '1.1.1-rc.2', '1.1.1-rc.1', '1.1.2-rc.1', '1.1.2-rc.2'':
- >1.0 would return 1.1.2-rc.2
- >1.1.1-rc.2 would return 1.1.2-rc.2
the acutal result is:
```
 > semver.maxSatisfying(['1.1.0', '1.1.1-rc.3', '1.1.1-rc.2', '1.1.1-rc.1', '1.1.2-rc.1', '1.1.2-rc.2'], '>1.0')
'1.1.0'
> semver.maxSatisfying(['1.1.0', '1.1.1-rc.3', '1.1.1-rc.2', '1.1.1-rc.1', '1.1.2-rc.1', '1.1.2-rc.2'], '>1.1.1-rc.2')
'1.1.1-rc.3'

// add 1.2.0 to the version list
> semver.maxSatisfying(['1.1.0', '1.1.1-rc.3', '1.1.1-rc.2', '1.1.1-rc.1', '1.1.2-rc.1', '1.1.2-rc.2', '1.2.0'], '>1.1.1-rc.2')
'1.2.0'
```
possiable to get the latest pre release based on a versin range? the answer is No.

https://docs.npmjs.com/misc/semver#prerelease-tags

If a version has a prerelease tag (for example, 1.2.3-alpha.3) then it will only be allowed to satisfy comparator sets if at least one comparator with the same [major, minor, patch] tuple also has a prerelease tag.

For example, the range >1.2.3-alpha.3 would be allowed to match the version 1.2.3-alpha.7, but it would not be satisfied by 3.4.5-alpha.9, even though 3.4.5-alpha.9 is technically “greater than” 1.2.3-alpha.3 according to the SemVer sort rules. The version range only accepts prerelease tags on the 1.2.3 version. The version 3.4.5 would satisfy the range, because it does not have a prerelease flag, and 3.4.5 is greater than 1.2.3-alpha.7.

The purpose for this behavior is twofold. First, prerelease versions frequently are updated very quickly, and contain many breaking changes that are (by the author’s design) not yet fit for public consumption. Therefore, by default, they are excluded from range matching semantics.

Second, a user who has opted into using a prerelease version has clearly indicated the intent to use that specific set of alpha/beta/rc versions. By including a prerelease tag in the range, the user is indicating that they are aware of the risk. However, it is still not appropriate to assume that they have opted into taking a similar risk on the next set of prerelease versions.

semver.js source code: https://github.com/npm/node-semver/blob/v5.5.0/semver.js#L1121

how to get the latest pre-release verions?

dist-tag: https://docs.npmjs.com/cli/dist-tag
```
Publishing a package sets the latest tag to the published version unless the --tag option is used. For example, npm publish --tag=beta.

By default, npm install <pkg> (without any @<version> or @<tag> specifier) installs the latest tag.
```
so, to release a new pre-release version:
```
npm publish --tag next
```
to use the latest pre-release verion:
```
npm install <package>@next
```
to verify:
```
npm dist-tag ls -a <package>
```

« Previous Page: 4 of 7 Next »

Relaxed Binding of Type-safe Configuration Properties in Spring Boot 1.5

Question: what’s the output?

What? I found a bug in Spring Boot?

Conclusions

在新加坡申请泰国旅游签证

Run Selenium with Headless Chrome in Docker(CentOS)

build image

execute selenium from python

docker image & source code

troubleshootings

Public key for google-chrome-unstable-x.rpm is not installed

ERROR Message: unknown error: DevToolsActivePort file doesn’t exist

'Software Testing Foundations' Notes

The Fundamental Test Process

Test Planing and Control

Test Analysis and Design

Test Implementation and Execution

Test Evaluation and Reporting

Test Closure Activities

General Principles of Testing

Test Plan

Test Entry and Exit Criteria

npm versioning

npm versioning

update dependencies based on semantic versions

prerelease tags and ranges

how to get the latest pre-release verions?