Changelog federated by tensorflow – pyup.io (2023)

Main features and improvements

* Improved serialization method to add structures larger than 2 GiB (~500
million model parameters):
* `tff.learning.models.function model`
* `tff.programme.FileProgramStateManager`

Troubleshooting

* Fixed a bug with `copy.deepcopy` for expectable (unselectable) structs
em `tff.learning.programs`.
* Fixed an error when continuing an assessment
"tff.learning.programs.EvaluationManager" where the evaluation restarted
would overwrite the posted metrics.

Main features and improvements

* Reduced memory usage for entropy compression.
* Version `com_google_protobuf` updated to `v3.19.0`.
* Removed dependency on `six`.

last minute changes

* Removed the default value for the abstract base class key parameter
`tff.program.ReleaseManager`.

Troubleshooting

* Fixed a syntax issue with spaces when shutting down a process when using
localhost C++ execution context.
* Changed `tff.simulation.build_uniform_sampling_fn` to output
increases for non-integer entries.
* Just expect a subprocess instance if it's not None.

Main features and improvements

* TFF-C++ Runtime now installed by default. Note that this version has a
significantly larger PIP packet size.
* Introduce `tff.learning.programs` into the federated program logic by using the
APIs `tff.program`.
* Updated `tensorflow` to version `2.11.0`.
* Updated `tensorflow_compression` to version `2.11.0`.
* Updated `bazel_skylib` to version `1.3.0`.

Main features and improvements

* Ignore model updates that are not finite
`tff.learning.templates.build_apply_optimizer_finalizer`.

last minute changes

* Removed deprecated APIs in `tff.learning.framework`
* Update Python package scripts to use Python 3.10 by default.
* Remove module placeholder imports from **init**.py files in TFF.
* Update Python package scripts to use Python 3.10 by default.

Troubleshooting

* Remove `functools.wraps` from `tff.tf_computation`.
* Fixed typo in iNaturalist dataset docstring.

Main features and improvements

* Added support for `tff.learning.models.FunctionModel` to all methods in
`tff.learning.algorithms`.
* Added support for `tf.data.DataSpec` for `tff.types.infer_unplaced_type`.
* Use a `tensorflow::ThreadPool` for concurrency in C++
`TensorFlowExecutor`.
* A new `tff.experimental_tf_fn_computation` trace decorator has been introduced that
uses `FunctionDef` instead of `GraphDef` tracing and provides `tf.function`
automatic control dependencies.
* Renamed `number_of_clients` to `num_clients` in the composite program API.
* The following API has been replaced by the Compositors API in `tff.learning.templates`.
* `tff.learning.framework.build_model_delta_optimizer_process`
* `tff.learning.framework.ClientDeltaFn`

Troubleshooting

* Fixed bug in "Client Efficient Big Model Federated Learning".
Tutorial on how to use correct dense form.

Main features and improvements

* Added `tff.learning.models.FunctionalModel` support for
`tff.learning.algorithms.build_mime_lite`.
* Updated `tensorflow-privacy` to version `0.8.6`.
* Added an abstract interface describing an asynchronous context
* Removed references to `tff.framework.Context`.
* Reference to `tff.simulation.datasets.gldv2.get_synthetic`.
* Added prefetching data source in `tff.program.PrefetchingDataSource`.

last minute changes

* Deleted obsolete
`tff.learning.framework.build_encoded_broadcast_process_from_model`.
* The alias `tff.learning.ModelWeights` is deprecated
`tff.learning.framework.ModelWeights`, now changed to
`tff.learning.models.ModelWeights`. The code must be updated before the next one
Start.

Troubleshooting

* Fixed a variable order bug when creating metrics
`tff.learning.models.functional_model_from_keras`.
* Improved `tff.tf_computation` tracing to also trace `functools.partial`
objects.

Known bugs

* Colab Compatibility: TFF requires Python 3.9 while the Colab runtime uses Python
3.7.

Main features and improvements

* Added Python 3.10 support.
* Improved support for `numpy` values ​​in `tff.program` API.
* The dataset serialization size limit has been increased to 100 MB.
* Added new method `tff.learning.ModelWeights.convert_variables_to_arrays`.
* Added new aggregation factories for metrics in `tff.learning.metrics`.
* Parallele Aggregation in `tff.framework.ComposingExecutorFactory`.

last minute changes

* Updated to use `jax` and `jaxlib` version `0.3.14`.
* Renamed `tff.program.CoroValueReference` to
`tff.program.AwaitableValueReference` to reflect the relaxed contract.

Troubleshooting

* Improved documentation for `tff.simulation.build_uniform_sampling_fn`,
`tff.learning.robust_aggregator`,
`tff.aggregators.PrivateQuantileEstimationProcess`.
* Error in the tutorial documentation "High Performance Simulation with
Kubernetes“.
* Fixed bug adding momentum hyperparameters to SGDM optimizer when
Pulse has been set to 0.
* Removed assertion that preprocessed records in a
`tff.simulation.baselines.BaselineTask` have the same element structure.
* Fixed a memory leak when moving numpy arrays between Python and C++
does not constrain executor C++.
* Fixed bug in composite program API when using multiple release managers
share the same value.

Thanks to our employees

This release includes contributions from many people at Google, as well as:
Madhava Jay, nbishdev

Main features and improvements

* Added support for `tff.learning.models.FunctionalModel` for
`tff.learning.algorithms.build_fed_sgd` e
`tff.learning.algorithms.build_fed_prox`.
* Increased gRPC message limit from 1GB to 2GB.
* Added hyperparameter getters/setters to various components in tff.learning.

last minute changes

* Updated `tensorflow` to version `2.10`.

Troubleshooting

* Improved documentation for
`tff.analytics.heavy_hitters.iblt.build_iblt_computation()`.
* Fixed wrong docstring of `tff.federated_select`.
* Fixed typo in federated program example.

Main features and improvements

* Added get/set_hparams methods for `tff.learning.templates.ClientWorkProcess`.
* Reference to `tff.learning.algorithms.build_mime_lite_with_optimizer_schedule`.
* Updated `tensorflow-privacy` to version `0.8.5`.
* Reference to `tff.learning.entropy_compression_aggregator`.
* Reference to `tff.aggregators.EliasGammaEncodedSumFactory`.
* `tff.program.ClientIdDataSource` and added
`tff.program.ClientIdDataSourceIterator` to work with a data source of
federated client IDs.

last minute changes

* Removed backend iree prototype.
* Added new dependency on TensorFlow Compression.

Troubleshooting

* Fixed implementation of `loading_remote_data` tutorial.
* Corrected the docstring of
`tff.simulation.datasets.stackoverflow.get_synthetic`.

Known bugs

* TFF's Python 3.9 input conflicts with Colab's Python 3.7 runtime.

Main features and improvements

* Updated to use "Bazel" version "5.3.0".
* Updated the conventions used to specify the version of a Python dependency,
see https://github.com/tensorflow/federated/blob/main/requirements.txt for
More information.
* Updated setup.py to explicitly fail on pip install in Python 3.10.
This has always failed at runtime, but now explicitly fails on install
my `beep'.
* Updated loading_remote_data notebook content and added content for
`FederatedDataSource`.
* Added a `type_signature` TFF attribute for objects of type `MapReduceForm`.
* Added one
[Serie](https://github.com/tensorflow/federated/blob/main/docs/design/TFF_101_lingua_federata.pdf)
[de](https://github.com/tensorflow/federated/blob/main/docs/design/TFF_102_executors.pdf)
[Folien](https://github.com/tensorflow/federated/blob/main/docs/design/TFF_103_transformations.pdf)
to the GitHub repository (so not part of the PIP package) where a
Dive deep into TFF.

last minute changes

* Updated tf-privacy version to `0.8.4`.
* Improved tf-model-optimization version to `0.8.3`.
* Removed `initialize` from `MapReduceForm`.
* `SequenceType` now automatically converts any `StructWithPythonType` that
contains a `list` to a `tuple` for compatibility with `tf.data`.
* The "model_fn" and "model" parameters have been unified
`tff.learning.algorithms.build_weighted_fed_avg`.
* `MapReduceForm` now accepts a `type_signature` argument in its constructor,
and no longer accepts an "initialize" argument.
* `MapReduceForm` no longer contains an `initialize` attribute.

Troubleshooting

* Relaxed overly strict type equivalency check for assignment in TFF-TF code
Generation.

Main features and improvements

* Estenda `tff.analytics.heavy_hitters.iblt` com API `create_chunker` para
Encoding of non-Unicode strings.
* Estender `tff.aggregators.DifferentiallyPrivateFactory.tree_aggregation` com
an optional `record_aggregation_factory` argument.

last minute changes

* Replaced `ModularClippingSumFactory` with `SecureModularSumFactory` in
`tff.analytics.build_hierarchical_histogram_process`.

Known bugs

*TFF's Python 3.9 input conflicts with Colab's Python 3.7 support.

Main features and improvements

* Add a MimeLite implementation that enables the optimizer's learning rate
plan on
`tff.learning.algorithms.build_mime_lite_with_optimizer_schedule`.

last minute changes

* None

Troubleshooting

* None

Known bugs

*TFF's Python 3.9 input conflicts with Colab's Python 3.7 support.

Main features and improvements

* Added `ReleaseManager`s to make authoring program logic more convenient.
* Updated the TFFs `attrs` dependency to version `21.4.0`.
* Updated TFF `tensorflow-privacy` dependency to version `0.8.1`.

last minute changes

* Changed `tff.learning.BatchOutput` from an attrs class to a namedtuple.
* Removed unused `tff.learning.framework.parameter_count_from_model` API.

Main features and improvements

* Add tests for `namedtuple`s in `tff.program` package.
* Add the `num_subrounds` parameter to the mergeable context to allow callers to do this
optional sequence sub-rounds.
* Added metrics support for `tff.learning.models.FunctionalModel` including
Updates to the helper function `create_functional_metric_fns` and the
Chamador Downstream `tff.learning.algorithms.build_weighted_fed_avg`.

Troubleshooting

* Fixed typos in types created to test the `tff.program` package.
* Fix some sample program typos.
* Fix tests that don't seem to run on CI.
* Fixed naming error for merged Python execution.
* Ensure that the exceptions thrown by the remote executor stub implement the gRPC error
Interface.
* Update `tff.structure.Struct` integration with pytrees JAX to avoid smoothing
the whole structure.
* Use Python 3.7 compatible type annotations until Colab is upgraded to Python
3.9.

Main features and improvements

* Update `MemoryReleaseManager` to save `type_signature` on release
Values.
* Add a type_signature parameter to the ReleaseManager.release method.
* Unify retry logic between TFF-C++ and TFF-Python.
* Update TFF posts and collaboration links to reference Discord
Server.

last minute changes

* Move the python executor stacks file to `python_executor_stacks.py` in
`executor_stacks` directory.

Troubleshooting

* Make sure deserialization of the dataset returns irregular and sparse tensors,
as needed according to the TFF type of the data set.
* Make `metric_finalizers` use metric constructors if available.

Main features and improvements

* Updated tutorials on how to use `tff.learning.algorithms` API.
* Asynchronous TFF execution contexts no longer assume a single global
cardinality; they simultaneously invoke any computation that requires parallelism.
is necessary.

last minute changes

* removed `tff.learning.build_federated_averaging_process`; users must
Migre para `tff.learning.algorithms.build_weighted_fed_avg`.

Troubleshooting

* Clarification of semantics for TFF-C++ Multimachine `Dispose`, `DisposeExecutor`,
and executor switching to avoid the occurrence of exceptions and spam logs in the
course of normal operation.
* Fixed unsigned integer overflow for TFF-C++
`max_concurrent_computation_calls`.
* Normalizes the dominant invocation form before attempting to compile it
`MergeableCompForm` which removes spurious errors in dependent aggregation
Verification.

Known bugs

* Serialization/deserialization of tf.data.Datasets, resulting in non-dense result
Tensors for runtime on multiple machines can run into problems:
* `tff.framework.deserialize_value` may not be able to be deserialized
tf.data.Datasets Gerando RaggedTensors oder SparseTensors.
* `tff.framework.serialize_value` may not serialize tf.data.Datasets
Generating SparseTensors.

Main features and improvements

* New Colab notebook demonstrating how to use `DataBackend` for remote loading
records.
* Added a CreateDataDescriptor helper function.
* Added a working binary serving the TFF-C++ executor service.

Troubleshooting

* Fixed bugs with intermediate aggregation and controller crashes
Gras.

Main features and improvements

* TensorFlow updated to `2.9.1`.
* Update pybind11 to `2.9.2`.
* Re-enable cpp_fast_protos.
* Introduces the Container class to run coroutines on a dedicated thread, making it possible
TFF synchronous execution interfaces for use in conjunction with others
asynchroner Code.
* Use the latest version of TFF in Colab notebook links.
* Rename the helper functions that create test `MeasuredProcess`s.
* Add a compiler transform that checks tensorflow calculations against the list of
allowed operations.
* Specify return types explicitly in the `program` package.
* Adds convenience function to define a local asynchronous CPP execution context.
* Move jax components to a non-experimental namespace.

last minute changes

* Change the compiler flag `_GLIBCXX_USE_CXX11_ABI` to `1`.

Main features and improvements

* Adds error message logging to TFF C++ execution context.
* Adds test coverage for C++ runtime with aggregators.
* Resets the "Workers absent with fixed customers per round" test.
* Add full examples of using "DataBackend" with TFF compositions.
* Updated MapReduceForm documentation to include the two additional insurances
intrinsic sum.
* tff.learn
* Relax type checking in LearningProcess from just SequenceType
also allow SequenceType structures.

last minute changes

* Remove usage of `tff.test.TestCase`, `tff.test.main()` and delete
The `test_case' module.
* Update the test utility docstrings to use consistent vocabulary.
* Updated to TensorFlow 2.9.0
* Renomeie `compiler/test_utils` para `compiler/building_block_test_utils`.
* Remove any unnecessary use of `pytype:skip-file`.
* Specify the "None" return type of "ReleaseManager.release".
* Remove use of deprecated numpy types.
* Replace obsolete `random_integers` with `randint`.

Troubleshooting

* Fixed numpy warning.

Main features and improvements

* Added "asyncio.run" call to Metrics Manager startup calls to ensure this
compatibility with
https://github.com/tensorflow/federated/commit/a98b5ed6894c536549da06b4cc7ed116105dfe65.
* Added example and documentation for the Federated Program API.
* Improved `model_update_aggregator` to support mixed float structures
dtypen.
* Create a mirror of
`tff.simulation.compose_dataset_computation_with_iterative_process` para
`tff.learning.templates.LearningProcess`.
* Added logging of expected sequential calculations for local TFF-C++ runtime.

last minute changes

* Moved declarations from `tff.test.TestCase` to `tff.test.*` as functions.
* Removed `assert_type_assignable_from` feature.
* Movido `assert_nested_struct_eq` for the `type_conversions_test` module.
* Removed comparisons of `client_train_process` and fedavg_ds_loop.

Troubleshooting

* Fixed comparisons with enums in the benchmarks package.
* Fixed async_utils.SharedAwaitable exception trigger.
* Fixed several lint bugs.

Main features and improvements

* Veraltet von `tff.learning.build_federated_averaging_process`.
* Added API to convert `tf.keras.metrics.Metric` to a pure set
`tf.functions`.

last minute changes

* Renomeado `ProgramStateManager.version` para `ProgramStateManager.get_versions`.

Troubleshooting

* Fixed the "datasets/" path in the TFF ClientData tutorial.

Main features and improvements

* Updated .bazelversion to `5.1.1`.
* Updated `tff.program` API to use `asyncio`.
* New APIs in `tff.framework` package:
* `tff.framework.CardinalitiesType`.
* `tff.framework.PlacementLiteral`.
* `tff.framework.merge_cardinalities`.
* `tff.analytics`
* Added new `analytic_gauss_stddev` API.

last minute changes

* Renomeado `ProgramStateManager.version` para `ProgramStateManager.get_versions`.

Troubleshooting

* Fixed some Python lint bugs related to Python 3.9 linting.
* Cleanup of obsolete TODOs throughout the codebase.

Known bugs

* Version 0.21.0 cannot be imported at this time
[colab](https://colab.research.google.com) if the Python version is smaller
than Python 3.9. Please use a runtime environment with a Python version greater than
Python 3.9 or use TFF version 0.20.0.

Main features and improvements

* `tff.analytics`
* Added new aggregation factory "tff.analytics.IbltFactory".
* Added new IBTL tensor encoder/decoder libraries and uses them in
`tff.analytics.heavy_hitters.iblt.build_iblt_computation`.
* `tff.aggregator`
* Added as_weighted_aggregator to tff.aggregator.Factory API.
* `tff.learn`
* Improved compilation and execution performance of
`tff.learning.metrics.secure_sum_then_finalize` agrupando tensores por
D-Typ
* Added `set_model_weights` method and default implementation for
`tff.learning.templates.LearningProcess`.
* Added new reset_metrics attribute to tff.learning.Model.
* Note on `schedule_learning_rate` for `tff.learning.optimizers`.
* Added new `tff.learning.ddp_secure_aggregator` for distributed
Differentiated privacy.
* `tff.simulation`
* Added option to distort train images in CIFAR-100 baseline task.
* Changed the default string length for Shakespeare's baseline task to
a more reasonable value.
* Essential
* Alternative runtime to create new RemoteExecutors with different
Cardinalities instead of resetting the cardinality in the remote
Service.

last minute changes

* Removed support for Python 3.7 and 3.8, TFF supports 3.9 and later.
* Deprecated attributes `report_local_outputs` and removed
`federated_output_computation` de `tff.learning.Model`
* Removed `Ingest` method from `tff.Context`

Troubleshooting

* Various typos in tests, code comments and pydoc.

Known bugs

* Sequences (records) of SparseTensors do not work in the C++ runtime.
* Calculations when the cardinality of `CLIENTS` is zero do not work in Python
Duration.
* Assigning variables to a Keras model after construction within a `model_fn`
yields a non-deterministic graph.

Main features and improvements

* Added `tff.program` API; This API is still under active development, but may be
used to compose common and platform-specific: program logic, components and
Privacy concepts for creating federated programs.
* Added Python 3.9 support.
* Added datasets from CelebA and iNaturalist to `tff.simulation.datasets`.
* Added tff.analytics API for federated analytics including heavy private data
refutation algorithms.
* Added `tff.learning.algorithms` API, including TFF implementations of
FedProx, FedAvg with learning rate planning, federated k-means and
MimeLite.
* Added `tff.learning.metrics` API to support easy configuration of
Aggregation of metrics across clients via new `metrics_aggregator` argument.
* Added argument `metrics_aggregator`
`tff.learning.build_federated_averaging_process` e
`tff.learning.build_federated_evaluation`.
* Added methods `report_local_unfinalized_metrics` and `metric_finalizers` for
`tff.learning.Model` and obsolete `report_local_outputs` e
`federated_output_computation`.
* Added `tff.learning.optimizers` API for purely functional build
Optimizers and implementations by SGD, Adagrad, Rmsprop, Adam, Yogi,
* Added `tff.learning.reconstruction` API for partial local build
federated learning algorithms, including federated reconstruction.
* Added `tff.learning.templates` API to support building learning algorithms
in a modular way.
* Added `tff.simulation.baselines` API to support learning assessment
Algorithms for a set of representative tasks.
* Reference to `tff.aggregators.DifferentiallyPrivateFactory.tree_aggregation` para
supports o [Algorithm DP-FTRL](https://arxiv.org/abs/2103.00039).
* Addictionado `tff.aggregators.SecureModularSumFactory`
* `tff.aggregators.DiscreteFourierTransformFactory` and added
`tff.aggregators.HadamardTransformFactory` to support rotation based
aggregators.
* Added `tff.aggregators.concat_factory` to show structures as
single tensioner.
* Note on `tff.backends.native.create_mergeable_comp_execution_context`,
`tff.backends.native.set_mergeable_comp_execution_context`; estes podem ser
Used with a distributed runtime to scale to tens of thousands of clients.
* Improved performance of many `tff.simulation.datasets.ClientData`
subclasses.
* Added `tff.simulation.datasets.ClientData.serializable_dataset_fn`
-Attribute that allows the creation of datasets within TF/TFF calculations.
* Added debug_measurements option to aggregators in tff.learning.
* Added support for unique null-client aggregations.
* Added support for Python data classes as function and return parameters
Values ​​for TFF calculations.
* Added automatic insertion of `tff.federated_zip` to invocation of
federated custom TFF calculations.
* Added utilities to `tff.simulation.datasets` to store federated datasets
to a `tff.simulation.datasets.SqlClientData` compatible SQL database.
* Added `tff.learning.models.FunctionalModel` and
`tff.learning.models.functional_model_from_keras`.
* Increased maximum flow of tensioners. Tensors are now flowing here, there and everywhere.
* Updated Python dependencies:
* Updated `absl-py` to version `1.0.0`.
* Updated "attrs" to version "21.2.0".
* Added "Farmhashpy" version "0.4.0".
* Updated `jax` to version `0.2.27`.
* Updated `jaxlib` to version `0.1.76`.
* Updated `numpy` to version `1.21.4`.
*Repeat removed.
* Updated `tensorflow-model-optimization` to version `0.7.1`.
* Updated `tensorflow-model-optimization` to version `0.7.3`.
* Updated `tensorflow` to version `2.8.0`.
* Added support for building many dependencies, including using "tensorflow".
Basel.
* Updated Bazel dependencies:
* Updated `rules_python` to version `0.5.0`.
* Updated `com_google_protobuf` to version `v3.18.0-rc1`.
* Added `absl_py` version `1.0.0`.
* Added `com_google_googletest` version `release-1.11.0`.
* Added `io_bazel_rules_go` version `v0.29.0`.
* Added `bazel_skylib` version `1.0.3`.
* `pybind11_abseil` hinzugefügt.
* Added `pybind11_bazel`.
* Added `pybind11_protobuf`.
* Added `com_google_absl` version `20211102.0`.
* Added `tensorflow_org` version `v2.8.0`.

last minute changes

* Removed support for Build Source on macOS.
* Removed Python 3.6 support.
* Removed symbol "tff.framework.type_contains", use "tff.types.contains".
rather than.
* Removed many symbols from `tff.simulation`, see them at
`tff.program` instead.
* Removed support for converting non-OrderedDict mapping types to
`tff.Value`s.
* Remove `tff.simulation.datasets.ClientData.from_clients_and_fn` in favor of
`tff.simulation.datasets.ClientData.from_clients_and_tf_fn`.
* `tff.simulation.datasets.ClientData.preprocess` limited to support
TF serializable functions.
* Removed `tff.backends.reference` and the context reference it contained.
* Remove `tff.learning.build_federated_sgd_process` in favor of de
`tff.learning.algorithms.build_fed_sgd`.
* Removed `tff.simulation.run_simulation` in favor of
`tff.simulation.run_training_process`.
* Remove `tff.learning.framework.EnhancedModel`.
* Entfernen Sie `tff.learning.framework.build_stateless_mean`.

Troubleshooting

* Fixed broken links in the documentation.
* Fixed many pytype bugs.
* Fixed some inconsistencies in Bazel visibility.
* Fixed bug where `tff.simulation.datasets.gldv2.load_data()` would result
a mistake.

Main features and improvements

* Introduced new intrinsics: `federated_select` and `federated_secure_select`.
* New `tff.structure_from_tensor_type_tree` to help with structure handling
`tff.TensorType` in value structures.
* Many new factory implementations of `tff.aggregators`.
* Introduced the `tf.data` concept for data URIs.
* New `tff.type` package with utilities to work with `tff.Type` values.
* Initial experimental support for `tff.jax_computation`.
* Extend `tff.tf_computation` support for `SpareTensor` and `RaggedTensor`.

last minute changes

* Update gRPC dependency to 1.34.
* Moved `ClientData` interface and implementations
`tff.simulation.datasets`.
* Renomeado `tff.utils.update_state` para `tff.structure.update_struct`.
* Removed `tff.utils` namespace, migrated all icons, many after
`tff.aggregators`.
* Infinite EMNIST record moved to federated search repository.
* Removed `rpc_mode` argument for remote executors along with streaming mode.
* Remove obsolete from `tff.federated_apply`.
* Removed `tff.federated_reduce`, all users can use
`tff.federated_aggregate`.
* Removed dependency on pip `HDF5ClientData` and `h5py`.
* Removed `setattr` functionality in `tff.ValueImpl`.

Troubleshooting

* Improved `tf.GraphDef` comparisons.
* Force close generators used to send functions to compute wrappers,
Avoiding race conditions in Colab.
* Fixed asynchronous use of trace libraries for Python3.9 compatibility.
* Fixed problem with pool destruction and built-in `abc` type destruction.
* Fixed type intern for tensors with unknown dimensions.
* Fixed irrational consumption of `ClientData.create_dataset_from_all_clients`
Amount of memory/processing time.

Main features and improvements

* The `tff.simulation` package has been extended to add many new tools to run
Simulations (checkpoints and metrics manager, customer sampling functions).
* A number of new aggregators have been added to the `tff.aggregators` package
The factory.
* Added tff.structure API to expose 'Struct' class and
functions.
* Added `tff.profiler` API to provide useful functionality related to profiling.
* Added tff.backends.test package to show backends that are in focus
specifically testing a way to test a calculation using a
Integrated `federated_secure_sum`.
* Added tff.experimental package to expose less stable APIs.

last minute changes

* Replace a class base abstract `tff.aggregators.AggregationProcessFactory`
com `tff.aggregators.UnweightedAggregationFactory` e o
Klassen `tff.aggregators.WeightedAggregationFactory`.
* Replaced class `tff.aggregators.ZeroingFactory` with one
Function `tff.aggregators.zeroing_factory` with the same input arguments.
* Replaced class `tff.aggregators.ClippingFactory` with one
Function `tff.aggregators.clipping_factory` with the same input arguments.
* Updated `tensorflow` package dependency to `2.4.0`.
* Updated `absl-py` package dependency to `0.10`.
* Updated `grpcio` package dependency to `1.32.0`.
* Added a `jaxlib` package dependency on `0.1.55`.
* Updated `numpy` package dependency to `1.19.2`.
* Updated tensorflow-addons package dependency to 0.12.0.
* Updated tensorflow-model-optimization package dependency to 0.5.0.

Troubleshooting

* Fixed intrinsic handling of federated types of `sequence_reduce`.

Main features and improvements

* New `tff.aggregators` package with state aggregation interfaces
compositions.
* New Google Landmark dataset `tff.simulations.dataset.gldv2`
* New convenience APIs `tff.type_clients` and `tff.type_at_server`
* Reverse control of calculation methods for cleaner Python
Stack traces in case of error.
* Move executor creation to a factory default in executor service
Distributed runtimes to be independent of the number of clients.
* Significant improvements in type serialization/deserialization
* Nova API `tff.simulations.compose_dataset_computation_with_iterative_process`
to move the build execution from the client dataset to the leaves of the executor stack.
* Extend parameterization of `tff.learning.build_federated_averaging_process`
with the argument "use_experimental_simulation_loop" to make better use of multi-GPU
the settings.

last minute changes

* Removed `tff.utils.StatefulFn`, replaced by `tff.templates.MeasuredProcess`.
* Remove `tff.learning.assign_weights_to_keras_model`
* Finish removing `OptimizeDataset` operations from `tff.tf_computation`s.
* The `research/` directory has been moved
http://github.com/google-research/federated.
* Updates to `input_spec` argument for `tff.learning.from_keras_model`.
* Updated TensorFlow dependency to "2.3.0".
* TensorFlow model optimization dependency updated to 0.4.0.

Troubleshooting

* Fixed crash in streaming mode on remote executor.
* Wrap the `collections.namedtuple._asdict` calls into `collections.OrderedDict`
supports Python 3.8.
* Properly serialize/deserialize `tff.TensorType` with unknown shapes.
* Clean up TF lookup HashTable resources on TFF execution.
* Fixed Shakespeare record where OOV and last vocabulary character were the same
same.
* Fix TFF capture of Keras models with shared embeds.
* Fixed compile vulnerability for CanonicalForm.

Known bugs

* Tutorial "Federated Learning for Image Classification" fails to load
Plug-in `Projektor` para Tensorboard.
(https://github.com/tensorflow/federated/issues/914)
* Certain Keras models with activity regulation cannot be run
unrecoverable error (https://github.com/tensorflow/federated/issues/913).

Thanks to our employees

This release includes contributions from many people at Google, as well as:

Amitport, Ronaldseoh

Troubleshooting

* Fixed issue preventing Python lists from being "all_equal" values.

Main features and improvements

* User-supplied mirror types and minimizing the use of AnonymousTuple.

last minute changes

* Renamed "AnonymousTuple" to "Struct".

Main features and improvements

* Updated `tensorflow-addons` package dependency to `0.9.0`.
* Added API to expose native backend more conveniently. To see
`tff.backends.native.*` for more information.
* Added compiler argument to `tff.framework.ExecutionContext` API and
provided a compiler for the native execution environment that improves
The default TFF concurrency pattern.
* Introduced a new `tff.templates.MeasuredProcess` concept, a specialization
de `tff.templates.IterativeProcess`.
* Extended interfaces `tff.learning` for `tff.templates.MeasuredProcess`
Objects for aggregation and transfer calculations.
* Introduced new convenience method `tff.learning.weights_type_from_model`.
* Introducing the concept of `tff.framework.FederatingStrategy` which
parameters of the `tff.framework.FederatingExecutor` so that the
the implementation of a specific intrinsic is easier to deploy.
* Reduced duplication in ASTs generated by TFF.
* Enabled use of GPUs for remote workers.
* Documentation improvements.

last minute changes

* The return value of `IterativeProcess`
`tff.learning.build_federated_averaging_process` e
`tff.learning.build_federated_sgd_process` now compresses the second tuple output
(the metrics) to change the output of a federated value structure
to a federal value structure.
* The `tff.framework.set_default_executor` function has been removed, you should do this instead
Use o `tff.backends.native.set_local_execution_context` to make it more convenient
function or manually create a context and set it with
`tff.framework.set_default_context`.
* The base class `tff.Computation` now contains an abstract `__hash__` method,
to ensure build results can be cached. All custom implementations of
this interface should be updated accordingly.

Troubleshooting

* Fixed missing variable initialization issue for explicitly non-variables
added to each collection.
* Fixed the issue where table initializers would not run when the
The decorated function `tff.tf_computation` used no variables.

Thanks to our employees

This release includes contributions from many people at Google, as well as:

jvmcns

Main features and improvements

* Multiple TFF execution accelerators.
* New `tff.templates.MeasuredProcess` specialization of `IterativeProcess`.
* Further optimization of `tff.templates.IterativeProcess` ->
Compile `tff.backends.mapreduce.CanonicalForm`.

last minute changes

* Movido `tff.utils.IterativeProcess` para `tff.templates.IterativeProcess`.
* Removed `tff.learning.TrainableModel`, client optimizers are now arguments
kein `tff.learning.build_federated_averaging_process`.
* Required bump version of pip packages for Tensorflow (2.2), Numpy (1.18),
Panda (0.24), Graphics (1.29).

Troubleshooting

* Problem with GPUs in multi-machine simulations not being used and making errors
Deserialize datasets with GPU-assisted runtime.
* TensorFlow lookup table initialization fails.

Known bugs

* In some situations TF tries to insert records into tf.functions
via GPU device limits, which fails by default. This can be achieved through
certain uses of TFF,
[for example. as traced here] (https://github.com/tensorflow/federated/issues/832).

Thanks to our employees

This release includes contributions from many people at Google, as well as:

jvmcns

Troubleshooting

* Fixed issues in tutorial notebooks.

Main features and improvements

* Updated `absl-py` package dependency to `0.9.0`.
* Updated `h5py` package dependency to `2.8.0`.
* Updated `numpy` package dependency to `1.17.5`.
* Updated tensorflow-privacy package dependency to 0.2.2.

last minute changes

* Deprecated `dummy_batch` parameter from `tff.learning.from_keras_model`
Occupation.

Troubleshooting

* Fixed executor service issues with old executor API.
* Fixed remote executor testing issues with old executor API.
* Fixed issues in tutorial notebooks.

Main features and improvements

* Updated Tensorflow dependency from `2.0.0` to `2.1.0`.
* Updated Tensorflow addons dependency from 0.6.0 to 0.7.0.
* Updated Attr dependency from 18.2 to 19.3.
* Updated tfmot dependency from `0.1.3` to `0.2.1`.
* Added a federated CIFAR-100 dataset partition for
`tff.simulation.datasets.cifar100`.
* Made the high-performance parallel executor the default (replacing the
Reference executor).
* Added new `tff.learning.build_personalization_eval` to evaluate the model
Personalization Strategies.
* Added new federated intrinsic `tff.federated_secure_sum`.
* `tff.learning.build_federated_averaing_process()` agora leva um
`client_optimizer_fn` und `tff.learning.Model`.
tff.learning.TrainableModel is now deprecated.
* Improved performance in high performance executor stack.
* Implemented and exposed `tff.framework.ExecutorFactory`; All
`tff.framework...executor_factory` calls now return an instance of it
Class.
* Added `remote_executor_example` binary demonstrating the use of
Remote Executor in multi-machine deployments.
* Added `close()` method to executor, which allows subclassing
release resources.
* Updated documentation and scripts for building TFF Docker images
Duration.
* Automatically calls `tff.federated_zip` for entries for other federated
intrinsisch.

last minute changes

* Deprecated support for Python2.
* `tff.framework.create_local_executor` (and similar methods) renamed to
`tff.framework.local_executor_factory`.
* Deprecated `federated_apply()`, use `federated_map()` for all instead
placements.

Troubleshooting

* Fixed issue where different instances of the same model were different
named types. `tff.learning.ModelWeights` no longer names tuple fields
returned for model weights instead of relying on an ordered list.
* `tff.sequence_*` on non-enclosed types now correctly returns a `tff.Value`.

Known bugs

* The `tff.sequence_*`.. operations are not yet implemented in the new one
High performance executor stack.
* A subset of previously allowed Lambda traps is no longer supported by
the new execution stack.

Main features and improvements

* Python 2 support is now deprecated and will be removed in a future release.
* `federated_map` now works with `tff.SERVER` and `tff.CLIENT`
placements.
* `federated_zip` has received significant performance improvements and is now working
recursively.
* Added retry logic to gRPC calls in the runtime stack.

last minute changes

* `collections.OrderedDict` is now required in many places instead
Standard Python dictionaries.

Troubleshooting

* Fixed calculation of number of samples when Keras uses multiple
Starter.
* Fixed assumption that `tff.framework.Tuple` is returned
`Iterative process.next`.
* Fixed wrapping arguments in polymorphic calls in the new Executor API.
* Fixed support for `dir()` in `ValueImpl`.
* Fixed several issues in Colab/Jupyter notebook tutorials.

Troubleshooting

* Updated to use `grpcio` `1.24.3`.

Main features and improvements

* Add a federated_sample aggregation used to collect a sample
Client values ​​on the server using reservoir sampling.
* Updated to use `tensorflow` `2.0.0` and `tensorflow-addons` `0.6.0`
of the corresponding nightly package in `setup.py` to release the TFF
Python-Pakete.
* Updated to use "tensorflow-privacy" "0.2.0".
* Added support for type annotations of `attr.s` classes.
* Updated streaming method Execute in tff.framework.ExecutorService
asynchronous.
* PY2 and PY3 compatibility.

Main features and improvements

* TFF is now fully compatible and dependent on TensorFlow 2.0
* Add state aggregation with differential data protection with TensorFlow Privacy
(https://pypi.org/project/tensorflow-privacy/).
* Added state aggregation with compression using TensorFlow model
Optimization (https://pypi.org/project/tensorflow-model-optimization/).
* Improved executor stack for simulations, documentation and scripts for
Start simulations in GCP.
* New libraries for creating synthetic IID and non-IID datasets in simulation.

last minute changes

* "Examples" package divided into "Simulation" and "Research".

Troubleshooting

* Several improvements to error message order.
* Fixed dataset serialization for V1/V2 datasets.
* `tff.federated_aggregate` supports `accumulate`, `merge` and `report`
Methods with signatures containing tensors with undefined dimensions.

Main features and improvements

* Executor stack improvements: caching, deduplication, bi-directional
Streaming mode, ability to specify physical devices.
* Components for integration with custom MapReduce backends
(`tff.backends.mapreduce`).
* Improvements to simulation dataset APIs: ConcreteClientData, Random Seeds,
Stack overflow record, updated documentation.
* Utilities for encoding and various types of aggregation.

last minute changes

* Removed support for the deprecated `tf.data.Dataset` string iterator identifier.
* Increases required versions of grpcio and tf-nightly.

Troubleshooting

* Corrections in notebooks, typos, etc.
* Various fixes to adapt to TF 2.0.
* Fixed thread cleanup when exiting process in high performance executor.

Thanks to our employees

This release includes contributions from many people at Google, as well as:

Gui-U, Krishna Pillutla, Sergii Khomenko.

Main features and improvements

* Powerful simulation components and tutorials.

last minute changes

* Refactoring/consolidation in utility functions in tff.framework.
* Changed some of the tutorials on new executor stack components that are unique to PY3.

Troubleshooting

* Includes the "examples" directory in the pip package.
* Pip installed for TensorFlow and TFF in tutorials.
* Patches for asyncio in tutorials for use in Jupyter notebooks.
* Compatibility issues with Python 3.
* Support for `federated_map_all_equal` in reference runner.
* Adds missing implementations of generic constants and intrinsic operators.
* Fixes missing link in readme compatibility section.
* Adds some of the missing intrinsic reductions.

Thanks to our employees

This version includes contributions from many people on Google.

Main features and improvements

* Support for multiple outputs and loss functions in "Keras" models.
* Support for federated stateful broadcast and aggregation functions
Intermediate and federated SGD APIs.
* Extended `tff.utils.update_state` to handle more general `state` arguments.
* Added `tff.utils.federated_min` and `tff.utils.federated_max`.
* Shuffle `client_ids` in `create_tf_dataset_from_all_clients` standardmäßig auf
help optimization.

last minute changes

* Added dependencies to requirements.txt; in particular "grpcio" and
"harbor picker".

Troubleshooting

* Remove a dependency on `tf.data.experimental.NestedStructure`.

Thanks to our employees

This release includes contributions from many people at Google, as well as:

Dheeraj R. Reddy, Squadrick.

Main features and improvements

* Removed source-level TF dependencies and changed from `tensorflow` to
`tf-nightly` dependency.
* Added support for the `attr` module in the TFF type system.
* Introducing the new `tff.framework` interface layer.
* New AST transformations and optimizations.
* Keep using Python container in `tff.tf_computation`.

Troubleshooting

* Updated TFF model to reflect Kera's `tf.keras.model.weights` order.
* Multi-input Keras model. 416

Main features and improvements

* New `tff.simulation.TransformingClientData` API and associated infinity
EMNIST dataset (see http://tensorflow.org/federated/api_docs/python/tff for
Details)

last minute change

* `func` normalized to `fn` in the repository (renaming some parameters and
functions)

Troubleshooting

* Packed Keras templates can now be used with
`tff.learning.build_federated_evaluation`
* Keras models with non-trainable variables in middle layers (e.g.
BatchNormalization) can be reassigned with Keras models
`tff.learning.ModelWeights.assign_weights_to`

last minute changes

* Renomeie `tff.learning.federated_average` para `tff.learning.federated_mean`.
* Rename 'func' arguments to 'fn' throughout the API.

Troubleshooting

* Various fixes for typos in documentation and configuration scripts.

Main features and improvements

* Updated to use TensorFlow version 1.13.1.
* Federated SGD in `tff.learning.build_federated_sgd_process()` implementiert.

last minute changes

* "next()" function of "tff.utils.IteratedProcess" returned by
`build_federated_*_process()` no longer unpacks single-valued tuples (always
returns a tuple).

Troubleshooting

* Modify setup.py to require TensorFlow 1.x and do not upgrade to 2.0 Alpha.
* Stop unpacking single-valued tuples in the `next()` function from returned objects
von `build_federated_*_process()`.
* Clear cached Keras sessions when grouping Keras models to avoid references
outdated graphics.