Cadix Development Team

Atlas 0.2.2

2021-02-10T20:21:00+00:00

Atlas 0.2.2 implements new features introduced with Bombe 0.3.5, and makes it easier to use Atlas to merely analyse artefacts.

JarEntryTransformer#additions() is now supported
It is now possible to just ‘process’ a JAR file, where writing it to file is not desired.

Bombe 0.3.5 and 0.4.4

2021-02-09T17:48:00+00:00

Bombe 0.3.5 and 0.4.4 are small releases introducing some new APIs to bolster the capabilities of the jar transformation framework, namely allowing entries to be introduced. To accomplish this, a JarEntryTransformer#additions() method has been introduced.

The remapping transformer will additionally strip signature files and entries in the manifest. This transformer may in future be available standalone.

Bombe 0.3.5

The Jars utility has been deprecated in this version, advising consumers to switch to Atlas. Jars was removed in 0.4.0, so this just serves as a final notice to any lingering applications using the utility. Despite the deprecation, support for the new additions API has been implemented.

Lorenz 0.5.7

2021-02-01T15:12:00+00:00

Fixes

CSRG and TSRG now ignore package mappings, rather than erroneously reading them as class mappings.
MappingSet#deobfuscate(FieldType) will now correctly de-obfuscate object types of inner classes, where the parent class has a mapping - but the inner class does not.
GH-29: Avoid inheriting field mappings from a parent where the child class has a field of its own, of the same signature.

Atlas 0.2.1

2021-01-30T02:40:00+00:00

Atlas 0.2.1 provides some quality-of-life features designed to make Atlas more configurable and accountable.

Execeptions will no longer be dismissed when transforming, and rather rethrown (as was my intention).
Output jars will be repackaged to ensure that the manifest is read by JarInputStream correctly.
Atlases can specify their own ExecutorService to use, or their parallelism level if using our provided constructors.

Lorenz 0.5.6

2020-11-27T20:37:00+00:00

Configurable Parallelism during mapping merges

The mapping merging system introduced in 0.5.4 merges mappings in parallel - which in quick testing can improve the merge time by half. There may be cases where the max number of threads needs to be controlled. One common example is debugging Lorenz itself, having multiple merge threads running at once can make debugging a lot more difficult.

This release just adds one new method to MergeConfig: getParallelism(). Set this value using the new withParallelism() method. Check the javadocs for more info.

Allow arbitrary indexes for parameter mappings

Method mappings may now contain parameter mappings for arbitrary indexes, rather than being constrained to between 0 and the number of parameters in the method signature. This is nice from a general flexibility perspective as Lorenz is only a container and isn’t intended for validating mappings, but also fixes the issue where Lorenz can’t read mappings which using 1-indexed method parameters for instance methods. With this change it’s up to the user to decide how to use parameter mappings, Lorenz doesn’t dictate anything one way or the other (just like the other mapping types).

Any code which worked with Lorenz before will still continue to work, as this change only removes constraints which used to be present. If some code was written which relied on the existing index checks then this will technically be a breaking change, as you’ll need to handle those checks yourself. That is likely to be a minor edge case however, so this is still considered a minor release.

Lorenz 0.5.5

2020-11-07T13:03:00+00:00

Writer configurations

Writer configurations are a powerful new feature that allows the output of mappings writers to be fine-tuned as consumers would like. Currently, this is limited to:

The sorting functions used to write mappings for classes, fields, and methods.
The API is open-ended, so there is potential for further additions to be made.

Writer configurations can be created fluently through use the provided builder, which includes our default implementations.

protected MappingsWriterConfig config = MappingsWriterConfig.builder()
        .classMappingComparator(Comparator.comparing(Mapping::getFullObfuscatedName))
        .fieldMappingComparator(Comparator.comparing(Mapping::getFullObfuscatedName))
        .methodMappingComparator(Comparator.comparing(Mapping::getFullObfuscatedName))
        .build();

They are applied to a mappings writer by use of the new #setConfig method, and can be retrieved through the corresponding #getConfig() method.

writer.setConfig(MappingsWriterConfig.builder().build());

Fixes

The Kin mapping writer now produces consistent output, sorting the mappings as per the provided comparators. Output is also now rid of ‘junk’ mappings (those without valid children mappings, or a de-obf mapping of their own).

licenser 0.5.0

2020-10-04T22:15:00+00:00

We have made the first release of licenser under the Cadix Development organisation, which brings the project up-to-date and ready for changes being made to Gradle in 7.0.

Usage

plugins {
    id 'org.cadixdev.licenser' version '0.5.0'
}

Future

In future, we would like to look at migrating our Gradle plugins over to Kotlin - a type-safe language still preserving the lack of great boilerplate using Java, that seems to be the direction that Gradle is heading towards.

This will be a good opportunity to review functionality in our plugins, and work with our communities to resolve long-term frustrations with our projects.

A more broad goal is to introduce further configuration to allow fine-tuning of our plugins across-the-board, sparked by lack therefor in gitpatcher.

Lorenz 0.5.4

2020-08-25T03:48:00+00:00

Lorenz 0.5.4 includes yet-more bug fixes, and a new (and vastly more powerful) mapping merging API.

The ProGuard mapping reader now also reads field types.

The Bombe dependency has been bumped to 0.3.4, resolving a NullPointerException that would occur when using the ClassLoaderClassProvider.

Merging

Lorenz now has a new mappings merging API which aims at fulfilling 3 main goals:

Improve the output of the merging operation significantly, supporting many more edge cases and handling merges with less complete mapping sets.
Allow configuring (to some degree) the merging process to control some aspects of the default merge process.
Provide an API which is extensible and modifiable at multiple levels to enable custom merge situations.

There are a limited number of possible edge cases one would expect to run into when merging, so we laid them all out and made decisions on how best to handle them in the default case. Due to the extensible API users can override and change any of the default behavioe as necessary. The default behavior is laid out as follows:

Left	Right	Output	Note
`A -> B`	`B -> C`	`A -> C`	Typical case, easiest to handle
`A -> B`	Missing	`A -> B`	Standalone mappings get copied
Missing	`B -> C`	`B -> C`	Standalone mappings get copied
`A -> B`	`X -> Y`	`A -> B` and `X -> Y`	This is no different than the 2 above cases with missing mappings on each side. This is just meant to be a further example that if two unrelated mappings are present, a standard merger won’t know how to handle them other than copying both.
`A -> B`	`A -> C`	`A -> C`	By default the right mapping is considered the “most up to date” mappings, so in the case where both mapping sets provide mappings for the same obfuscated name, the right mapping is used in the default implementation.
`A -> B` (types)	`B -> B` (types) and `A -> C` (names)	`A -> B` (types) and `A -> C` (names)	This is an example of a special case situation where the left mapping only maps types, then the second mapping set only maps members, but from the expectation that the first mapping set was already applied. The default implementation should handle this case correctly.

How to read the above chart: Each mapping operation is made up of a left side and a right side. The chart represents this with the directional arrow ->. Merge operations in the table apply to any individual mapping, not the whole mapping set. That is, a field name mapping merging with another field name, or a class name merging with another class name.

Handling merges with less complete mappings sets are what is intended by the last row.

Directionality

It may seem a natural goal for merging mapping sets is for the merge operation to be commutative - in other words, when merging A -> B and B -> C you would get the same result as when merging C -> B and B -> A (except, reversed).

This is true for most cases, but for some special cases - most notably duplicate mappings - this property just can’t be handled in any reasonable way. In these situations where a decision has to be made regarding whether to keep one mapping or another (and neither being a better option by itself) we use the position of the mapping as the heuristic for which one to keep.

When merging A -> B and B -> C the names you get in the output are C. Because of this, the right side of a merge is considered the more correct set of names (since it is the output). If the merge operation were to be reversed A would be preferred instead. Because of this it is important to consider which direction works best for you merge.

Other Scenarios

There’s also 2 other factors necessary to consider when merging mapping sets that we’ll go over:

Duplicate mappings are possible
Field mappings can have type signature

Handling Duplicates

A duplicate mapping is when both mapping sets define a mapping for the same obfuscated member. For example, if you were to merge the following mappings, they would be considered duplicated because they both share com.example.Example as the obfuscated name.

com.example.Example -> com.example.OtherExample1

com.example.Example -> com.example.OtherExample2

For the default merge implementation right mappings take priority for all duplicate merge scenarios. So in this case the mapping which would be present in the output mapping set would be:

com.example.Example -> com.example.OtherExample2

Handling Field Types

Field mappings can have types, but often don’t. Because Java doesn’t allow duplicate field names visible in any class it’s not always necessary to explicitly list the type of the field. The complication comes when merging mapping sets where one mapping set does and the other doesn’t contain field types. In this scenario looking up the field from one mapping set to another in the default case by simply using the field signature would fail, as the type descriptor is missing in one of the mapping sets.

The new merge system provides a configuration option for whether to handle this case. The default options is LOOSE - that is, first check for the signature, but also check for just the field name and use it if the field signature can’t be found. This can be switched to STRICT where all field mappings must match signatures exactly.

Merging Code and API

The new merging system is made up of 2 main classes:

The JavaDocs for each class go into considerable detail on exactly what each class is and how it’s used, but here’s a quick rundown:

MappingSetMerger is the entry point for the merge operation. It’s an interface with a default implementation which can be retrieved with MappingSetMerger.create(). It does the job of walking the class hierarchy of both mapping sets and finding the members of each side that should be merged together. It decides which members to include based on the MergeConfig provided in the call to create() (or just the default config if none was provided) and determines which individual case each merge is.

MappingSetMerger is available with an overridable default implementation in case it’s necessary to modify how the above cases are decided individually. It is designed to let a user override individual methods in order to change said logic when necessary, but the general idea is that in most cases you’ll never need to modify the default implementation of MappingSetMerger.

MappingSetMerger doesn’t handle actually merging the mappings, all it does is recognize which type of merge each merge is, and delegates the handling of that merge to MappingSetMergerHandler. This handler class has individual methods for each mapping case, and is a fully-default interface. This means you can implement MappingSetMergerHandler and override only the mapping cases you want to modify for your custom merge operation. The merge cases are all described by the method names (and explained in further detail in the JavaDoc), and look like:

FieldMapping mergeFieldMappings(
    final FieldMapping left,
    final FieldMapping strictRight,
    final FieldMapping looseRight,
    final ClassMapping<?, ?> target,
    final MergeContext context
);

FieldMapping mergeDuplicateFieldMappings(
    final FieldMapping left,
    final FieldMapping strictRightDuplicate,
    final FieldMapping looseRightDuplicate,
    final FieldMapping strictRightContinuation,
    final FieldMapping looseRightContinuation,
    final ClassMapping<?, ?> target,
    final MergeContext context
);

FieldMapping addLeftFieldMapping(final FieldMapping left, final ClassMapping<?, ?> target, final MergeContext context);

FieldMapping addRightFieldMapping(final FieldMapping right, final ClassMapping<?, ?> target, final MergeContext context);

If you only need some custom logic when the right mapping set provides a mapping that the left mapping set doesn’t provide, then you could just override addRightFieldMapping with your custom logic and leave everything else default. You wouldn’t need to worry about walking the hierarchy or copying of the other members or handling any other special cases.

This separation of concerns between determining what kind of merge each case is and actually handling the merge operation for each merge case hopefully will allow you to easily customize mapping merges for whatever custom situation you have.

Existing Code

Each of the existing merge() methods on each of the mapping classes have all been updated to use this new merge system. The methods haven’t been removed and are still binary compatible with any existing code you have which may be using it. The output of the merge may differ however, as the logic has been considerably modified as described above.

One Last Thing

The default implementation of MappingSetMerger merges top-level classes in parallel. This is okay because MappingSetMergerHandler is a stateless class, and MappingSetMerger is completely thread-safe. Due to this merge time fell by as much as half of the previous merge time when testing against Minecraft mapping sets.

Bombe 0.3.4 and 0.4.3

2020-08-19T10:26:00+00:00

Bombe 0.3.4 and 0.4.3 resolves a regression made when splitting up the previous systems in Bombe 0.2.x, specifically a NullPointerException that can occur when using ClassLoaderClassProvider.

Bombe 0.3.3 and 0.4.2

2020-08-11T20:05:00+00:00

Bombe 0.3.3 and 0.4.2 resolves an oversight when bumping to ASM 7, notably that our ClassProviderInheritanceProvider was still using ASM6 - this has now been resolved to use ASM7, in addition to a new constructor to allow for third-parties to specify which ASM API version they wish to use.